Intro Usab Final Project Ideas
R. G. Bias, INF 385P, Fall, 2010
10/6/2010
A Special Request
I have a friend, Dr. Clayton Lewis, immediate past chair of the Computer Science Dept. at
the University of Colorado, Boulder. Clayton is teaching a similar course this semester at
UC. We’ve got this grand scheme that we’re going to connect our classes via remote
usability testing. His class meets at a different time from ours, so it will have to be
asynchronous. Sooo, I would like for two to four of you to volunteer to be a test subject for
them THIS WEEK (last this week, running into next Monday). Even their testing will be
asynchronous (unmoderated) – they’ll ask you to go to a web site and carry out some tasks,
and the tool THEY’RE using (loop11 – which see at http://www.loop11.com/). I would
think that especially folks in the Evaluation group might be interested in this. And then, if
we wish, they’ll reciprocate and we can run some of them in remote usability testing, using
Webex and Morae and the telephone. THEN (if it all works just right), toward the end of
the semester we can have a little discussion about the pros and cons of each method.
So, if you wish to volunteer just shoot me an email (SOON!), and I’ll pass your contact info
along to the Boulder folks. Thanks.
Some final project ideas
You do NOT have to choose something from this list. But tell me what you’re thinking
about and whom you’re teaming up with. By, say, 10/13/2010, midnight.
-
-
http://www.austinhabitat.org/
http://www.caritasofaustin.org/
http://www.newlifetexas.org/
http://www.hillcountryride.org/site/PageServer
http://www.abcaus.org/
Indeed, here’s a place to find a bunch of nonprofits:
http://www.thecityofaustin.org/organization/index.html
Those of you who might be interested in clean energy or other business endeavors in
the Austin Technology Incubator might check this out – I have a contact or two
within the ATI -- http://www.ati.utexas.edu/index.html
I know the people running this, and they’re eager for some free usability help:
http://www.greenribbonschools.org/
Here are some thoughts on test plans:
Some are substantive, and some are just the take-‘em-or-leave ‘em stylistic considerations of
someone who is pretty queer about word choice.
-
-
-
-
-
-
-
-
Reminder – I’m doing a blanket IRB form for our whole class. If you wish to
subsequently publish your results (unlikely, if you’re testing a convenience sample),
you’ll have to complete a separate, individual IRB form, retroactively.
It is a good thing to describe in your test plan (and, again later in your report) HOW
you chose your particular tasks to test. If you just picked ‘em out of the air, using
your own common-sense understanding of the site, that’s fine. Just say so.
Note, when you write your test REPORT, you’ll likely start with your test plan, and
just turn all the verbs into past tense. Then add the results and discussion. It’s not
quite that simple, but close.
One concern is WHERE to start the test participants. Do you take ‘em to the home
page of the site being tested? Or is one of your goals to see if they look for that site,
and can find it if they do? If the latter, I think it is a fine idea to START the test by
NOT mentioning the site in question, but by asking a more general question. Here’s
an example. Say you were testing the New Life Institute site. This Institute provides
low- or no-cost mental health counseling. It would make sense to say, as the first
task, “OK, you have a friend in Austin who needs some psychotherapy, but he
doesn’t have much money, nor mental health insurance. You decide to go online to
see if you might find someplace for him to go for some help. Show me what you’d
do.” Then you see a) if they know about New Life Institute and go looking for it, b)
if they perform a more general search, and if so what terms they use, and c) if they
find the NLI site, in any case. THEN after that task (whether they were “successful”
or not), task 2 might be something like, “Someone has told you there’s this group in
town called New Life Institute. Here’s their home page. Go find what their fees
are.” (Or whatever.) Of course, take care not to mention NLI in the Informed
Consent form, or other pre-test materials. (Indeed, you MIGHT avoid saying “the
website” as it will imply to the test participants that there is ONE site you’re testing.)
I think this may be an artifact of the test plan templates some students have used in
the past, but give some thought to what measures you actually intend to employ.
Are you REALLY going to count “nonverbal expressions of frustration”? If yes,
fine. But I often see students listing a whole bunch of measures they intend to
collect, and then in the test report they list just time on task, error rates, and
particular errors.
As with the tasks, say how you arrived at “measureable usability criteria.” Why have
you decided to declare “failure” if people can’t complete the task in five minutes?
Because the stakeholders say that’s their nickel? Because you observed yourself
carrying out the task the first time, on this and on a competitor site, and that seemed
about right? Whatever. Just say so.
A student once asked me, do we tell the test participant that one of the observers is a
stakeholder in the site being tested (perhaps even one of the designers), or do we
hide that from them? I don’t think we hide it. Indeed, I think the right answer
might be to say “we” are involved in the design (so that it doesn’t differentially
influence the results when one of you is interacting with the test participants), but to
make it clear that you are interested in making the site better, and thus in their candid
responses.
OK, some writing nits:
o et al. is written just as such – no period after the et, yes period after the al. Et
is a whole word, meaning “and.” Al. is an abbreviation for “alia,” “others.”
o I’m the last person in the world who still cares about this, but “their” is a
plural pronoun. People use it to avoid sexist pronouns. But they use it
incorrectly – “The user . . . their opinion . . . .” The best way to avoid sexist
pronouns is to go plural, when you can – “The users . . . their opinions . . . .”
A relatively poor way is to go passive voice – “The opinion of the user . . . .”
I’m fine with “his/her” or “he or she,” though I know lots of people find
this awkward.
o When you use a bullet list, make sure each item has parallel sentence
structure (e.g., starts with the same part of speech) and completes the
sentence appropriately. I often see things like the following
 Our goals were to see if users could:
 Correctly navigate the site.
 Liked the site.
 Good aesthetics.
That makes my teeth hurt.
o I realize even Time magazine has gone away from the “serial comma” (the
comma after the penultimate noun in a series of three or more), but I think it
helps disambiguate sentences. Plus if it’s good enough for Strunk and White
(and on page 2!!– “Thus write ‘red, white, and blue.’”), it’s good enough for
me.
“Templates? Did you say templates?”
Check out the file “ut_plan.doc”
http://www.stcsig.org/usability/resources/toolkit/toolkit.html.
Here’s a thin one: http://www.lib.washington.edu/Usability/testplan.html
Here’s an actual plan I found online:
http://www.faculty.english.ttu.edu/5388/Resources/Blurr%20Test%20Plan.pdf
Hmm. See if this is anything: http://www.htmlcenter.com/blog/usability-testing-for-yourweb-site/
The point is, there’s no one right way. Offer enough detail so that someone else could
replicate your study, and the stakeholders can tell what you’re going to do. The test plan is
an excellent team communications vehicle. It gives the developers (or marketers, or sales
team, or management) a chance to say, “Oh, no, don’t test that task. We really need you to
look at this task.” Then you negotiate. THEN, once you present your results, teams can’t
say “Oh, we didn’t know you were going to test THAT. THAT is not even in the product
plan anymore.” Or some such.