What is known as Moderated User Testing?
A moderated test is a real-time interview with someone who tests your product or service (or the
concept, prototype, or design of that product or service). Such interviews can also focus on collecting
information about user behavior and decision-making.
This type of test can be done remotely or in person. If done remotely, as a Live Conversation, your
interaction with the contributors occurs in real time, in the same “virtual” space. Contributors connect
with you using a remote conferencing tool that allows you to communicate through VoIP, phone, chat,
or a combination of methods.
When to use a moderated test
Moderated usability tests are great when you would like to...
•
•
•
•
Ask individualized follow-up questions in real time.
Ensure that contributors do not get stuck while doing something technically difficult, like dealing
with a complex or low-fidelity prototype when testing concepts or ideas.
Ensure quiet contributors are more vocal since the moderator can prompt them to share what
they are thinking.
Observe body language and pick up on the subtleties of contributor behavior.
What is unmoderated testing?
An unmoderated test is when contributors complete predetermined tasks without the guidance of a
moderator. The contributor decides when and where they would like to complete the test and provides
feedback as they record their session.
The contributor thinks out loud as they work the tasks, and the researcher reviews the recorded
sessions at a later time. Unlike with a moderated test, the researcher does not communicate with the
contributor in real time; they may decide to ask follow-up questions as part of the test. Or, after viewing
the session, they may follow up with the contributor. Because modifications cannot be done in the
moment with the contributor, unmoderated tests may require more effort up front.
When to use unmoderated tests
Unmoderated tests are useful when you would like to...
•
Gather feedback quickly. A test can often be completed by User Testing contributors on the
same day!
•
•
Ensure you’re getting unbiased contributor feedback. Sometimes a moderator’s presence can
influence the response or behavior of a contributor, who may be inclined to want to please the
person conducting the test.
Complete other tasks while contributors work on their structured activities. This is possible since
a moderator does not need to be present during each session.
What is Card Sorting?
Card sorting is a technique that involves asking users to organize information into logical groups. Users
are given a series of labelled cards and asked to organize and sort them into groups that they think are
appropriate. Card sorting helps you to design an information architecture (IA), workflow, menu structure
or website navigation paths.
1. Open card sorting
Users are asked to organize cards into groups that they feel is appropriate to them. They are then asked
to name each of the groups they have created with a label that best describes that group. This method is
commonly used for new/existing information architectures or organizing products on a site or when
starting to create a new IA from scratch.
2. Closed card sorting
Users are provided with both the content cards as well as the category cards and asked to place the
cards in these given categories. This method is normally used when adding new content to an existing
site or gaining a second round of insights after an open card sort.
Card sorting can also be either remote or face to face:
3. Remote card sorting
When conducting a remote card sorting session a user will work independently via their own computers
to sort the cards that are provided through an online software tool. There are several online software
tool available on the market today which allow you to set up and distribute to as many users as you
require for the test. Some of the most common online software tools are Optimal Sort, Simple card
sort and Usability. These online software tools provide you with several ways to analyze the data. As the
test is conducted remotely there is no contact with the user so there is no way of understanding the
reasoning behind why the user has arranged the cards in a certain way.
4. Face to face card sorting
Face to face card sorting sessions is in person with an observer present. Users are provided with a set of
cards to move around and asked to talk through their thoughts and reasoning behind their decisions.
The observer can clarify any observations they are unsure of to gain a better insight into why the user
has made those choices.
Analyzing the data
Once collected, the data is analyzed to identify common trends. This is done by using cluster analysis
software or if the sample size is small, by inputting the data into a spreadsheet to see if there are any
patterns in the groupings or labelling.
Advantages of card sorting
•
•
It is simple and cheap to run card sorting sessions.
Quick to get research conducted and results back from users, especially if sessions are
conducted remotely.
• Provides a good insight about the users and how they would expect the information to be
organized.
Disadvantages of card sorting
•
•
•
Results can be varied and there could be no real pattern in the data.
Analyzing the results can be time-consuming depending on the complexity of the data
Card sorting does not take into account users’ tasks and goals, so if the research is used without
taking this into account, you could end up with a structure that is not usable for the user to
complete their tasks on the site.
What is tree testing?
Tree testing is a UX research method that allows you to evaluate the hierarchy and findability of topics
in a website or app. It's an important step in your research plan. In a tree test, participants are
presented with a text-only version of the site's hierarchy and asked to complete a series of tasks. The
goal boils down to answering the question, “Can users find what they are looking for?”
What is A/B testing?
A/B testing (also known as split testing or bucket testing) is a method of comparing two versions of a
webpage or app against each other to determine which one performs better. A/B testing is essentially
an experiment where two or more variants of a page are shown to users at random, and statistical
analysis is used to determine which variation performs better for a given conversion goal.
Running an A/B test that directly compares a variation against a current experience lets you ask focused
questions about changes to your website or app and then collect data about the impact of that change.
Testing takes the guesswork out of website optimization and enables data-informed decisions that shift
business conversations from "we think" to "we know." By measuring the impact that changes have on
your metrics, you can ensure that every change produces positive results.
Get started with Optimizely A/B testing.
Ready to start your first experiment? Optimizely Web Experimentation can help your set up many types of
tests in minutes.
How A/B testing works
In an A/B test, you take a webpage or app screen and modify it to create a second version of the same
page. This change can be as simple as a single headline, button or be a complete redesign of the page.
Then, half of your traffic is shown the original version of the page (known as the control) and half are
shown the modified version of the page (the variation).
Eye Track Testing
Each user activity in the job will be given a duration that indicates the average quantity of time it would
take an experienced user to complete it. (Sterba, 2017)