Rocket Surgery Made Easy: The Do-It-Yourself Usability Testing Workshop London 5 October 2011 © 2011 Steve Krug Public Service Announcement: Do not shake hands with this guy > © 2011 Steve Krug Who is this guy, anyway? Steve Krug (steev kroog) (noun) 1. Son, husband, father 2. Resident of Brookline, MA 3. Usability consultant Advanced Common Sense Me and a few well-placed mirrors Corporate motto: “It’s not rocket surgery™” Nice clients Lexus.com Bloomberg.com Technology Review © 2001 Steve Krug © 2011 Steve Krug I get to work at home © 2001 Steve Krug © 2011 Steve Krug I get to work at home © 2001 Steve Krug © 2011 Steve Krug I get to work at home © 2001 Steve Krug © 2011 Steve Krug My intention for today Give you some practice so you’re comfortable testing Try to answer all your questions so you have no reason left not to test © 2001 Steve Krug © 2011 Steve Krug This morning Why do usability testing? Steve does a demo test Six Maxims Writing tasks and scenarios Lunch © 2001 Steve Krug © 2011 Steve Krug This afternoon You do your first practice tests You do another practice test Assorted topics Lingering questions Giveaways and feedback © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2011 Steve Krug Ground rules Tell the driver to speak up, if necessary Interrupt ANYTIME with questions I’ll answer questions about anything except that brief period during the late 70’s © 2001 Steve Krug © 2011 Steve Krug Anybody here from out of town? Graphic designers Information architects Developers/programmers “Marketing” Usability ______ Project managers Writers/editors Check signers Other? Left-handed? © 2001 Steve Krug © 2011 Steve Krug Anybody here from out of town? Your experience with usability testing Have conducted tests (facilitator)? Have observed tests? Have read usability test reports? Your organization’s use of testing Never? Right before (or right after) product ships? Routine (several times during development)? Farmed out? Have your own lab (and white coats)? © 2001 Steve Krug © 2011 Steve Krug One more question What are the biggest obstacles to your doing testing? © 2001 Steve Krug © 2011 Steve Krug Who’s read the book(s)? DMMT? Rocket Surgery? Watched the video? Don’t worry, be happy, ask questions © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2011 Steve Krug So… Why usability testing? Ten years ago, I realized something… © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2001 Steve Krug © 2001 Steve Krug © 2001 Steve Krug © 2001 Steve Krug © 2001 Steve Krug © 2001 Steve Krug © 2001 Steve Krug © 2001 Steve Krug © 2001 Steve Krug © 2001 Steve Krug © 2001 Steve Krug “My ideal home page,” as told by… © 2001 Steve Krug © 2011 Steve Krug “My ideal home page,” as told by… © 2001 Steve Krug © 2011 Steve Krug And now, a live demonstration © 2001 Steve Krug © 2011 Steve Krug A brave volunteer? We’ll try an actual test It’s painless It’s brief You’ll get a round of applause when we’re done Qualifying criteria: Have used a Web browser English-speaking adult Doesn’t work on __________ During the test You are observers Jot down top 1 or 2 problems you observed © 2001 Steve Krug © 2011 Steve Krug Debriefing What were the most serious problems? Observed problems © 2001 Steve Krug © 2011 Steve Krug DIY usability testing (nutshell version) Three users You’ll find more than you can fix No lab or mirrors Set up a monitor in another room so the development team and stakeholders can watch Record with Camtasia or Morae (Techsmith.com) or CamStudio or various Mac products No stats, no exit questions, no faux validity No big honkin’ report – 2 page email Debrief over lunch © 2001 Steve Krug © 2011 Steve Krug The maxims Six of them Questions highly encouraged What seems like it might not work for you? © 2001 Steve Krug © 2011 Steve Krug A morning a month, that’s all we ask. © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2011 Steve Krug When this happens: Say this: You’re not absolutely sure you know what the user is thinking (see below). Something happens that seems to surprise them. For instance, they click on a link and go “Oh” when the new page appears. “What are you thinking?” “What are you looking at?” (for variety) “What are you doing now?” (e.g., if you think they’re being silent because they’re reading) “Is that what you expected to happen?” They’re trying to get you to give them a clue. (“Should I use the ___?”) “What would you do if you were at home?” The participant makes a comment, and you’re not sure what triggered it. The participant suggests concern that he’s not giving you what you need. “Was there something in particular that made you think that?” “No, this is very helpful.” The participant asks you to explain how something is supposed to work. (“Do these support requests get answered right away?”) “I can’t answer that right now, because we need to know what you would do when you don’t have somebody around to answer questions for you. But if you still want to know when we’re done, I’ll be glad to answer it then.” The participant seems to have wandered away from the task. “What are you trying to do now?” “What would you do if I wasn't here?” “This is exactly what we need.” © 2001 Steve Krug © 2011 Steve Krug Start earlier than you think makes sense. © 2001 Steve Krug © 2011 Steve Krug Incorrect thinking © 2001 Steve Krug © 2011 Steve Krug Correct thinking © 2001 Steve Krug © 2011 Steve Krug Recruit loosely and grade on a curve. © 2001 Steve Krug © 2011 Steve Krug Naturally, we need to test people who are just like our target audience. Representative users! … people who actually use our site. Real users! … people who are a lot like our users. © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2011 Steve Krug Make it a spectator sport. © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2011 Steve Krug Focus ruthlessly on a small number of the most important problems. © 2001 Steve Krug © 2011 Steve Krug The problem is, testing works If you’ve done any testing, you know uncovers lots of problems quickly This is part of the problem It takes far less resources to find problems than to fix them You can find more in a day than you can fix in a month © 2001 Steve Krug © 2011 Steve Krug Problems you can find with just a few test participants Problems you have the resources to fix © 2001 Steve Krug © 2011 Steve Krug Things I have learned It’s easy to get seduced into fixing the easier problems first As a result, the most serious usability problems often remain for a long time © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2011 Steve Krug When fixing problems, always do the least you can do™. © 2001 Steve Krug © 2011 Steve Krug Your motto When fixing usability problems, your motto should be: What’s the smallest change we can make that we think might solve the observed problem? © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug © 2011 Steve Krug Choosing tasks to test What do you have to show? Try not to let this limit your thinking, though You can get a long way with A sketch or a few “comps” Linked wireframes to test navigation HTML of a few pages that let you complete a task as long as you don’t stray See Carolyn Snyder’s Paper Prototyping © 2001 Steve Krug © 2011 Steve Krug Choosing tasks to test What do you wake up thinking about in the middle of the night? What tasks are crucial? …to the user and your business model Whenever possible, keep it real Free-range browsing tasks are a good thing Bad: “Buy a gift cupholder for under $35.” Better: “Order a book you’d like to have” © 2001 Steve Krug © 2011 Steve Krug Tasks vs. Scenarios Task: “Apply for a doctoral program at HBS” Scenario: “You’ve got an MBA, and after a lot of research you’ve decided to enter the doctoral program at HBS in Science, Technology & Management. Apply for admission to the program.” A scenario... Provides some context (“You are...”, “You need to...”) Just enough; DON’T get carried away Supplies specific information the user would actually have © 2001 Steve Krug © 2011 Steve Krug Writing the scenarios Don’t telegraph it Avoid using words that will appear on-screen Bad: “Customize your LAUNCHcast station.” Better: “Choose the kind of music you want to listen to.” Can be the hardest part Make it unambiguous Misunderstandings waste time and don’t [usually] produce useful insights Keep it short Trim any detail that doesn’t contribute Piloting the test will help © 2011 Steve Krug © 2001 Steve Krug Exercise 1: Listing tasks Jot down 3-7 of the most important tasks people need to do on your site (4 min.) E.g., “Find directions to your nearest Bank of America branch”, “Apply for a doctoral program” © 2001 Steve Krug © 2011 Steve Krug Exercise 2: Writing a scenario Choose one task from your list Write a scenario for this task (10 min.) A short paragraph or two “Don’t use Search” is OK Write it out on a piece of paper so you can read it verbatim © 2001 Steve Krug © 2011 Steve Krug Now send me your scenario(s) Go to tinyurl.com/********** Click the orange “Post a new message” button Use your name to sign in Type the URL you’re testing your scenario © 2001 Steve Krug © 2011 Steve Krug Exercise 3: Practice tests © 2001 Steve Krug © 2011 Steve Krug How it will work Pair up with someone Not from your company Each pair needs one laptop First 15 minutes: A is facilitator, testing his/her site B is participant Second 15 minutes: B is facilitator, testing his/her site A is participant © 2001 Steve Krug © 2011 Steve Krug Timing Get settled and I’ll tell you when to start Just follow the script! Welcome/instructions – 2 min. 5 questions – 2 min. Home page tour – 2 min The task – 7 min. I’ll give you a warning here Probing – 2 min. Wrap up – 30 secs. © 2001 Steve Krug © 2011 Steve Krug Tips for success Read the script verbatim No leading questions until probing time Keep them thinking aloud during task I can help if you have questions © 2001 Steve Krug © 2011 Steve Krug How’d it go? © 2001 Steve Krug © 2011 Steve Krug The debriefing Over lunch (or dinner, or breakfast) Right after the three test sessions Objective: Deciding what you’re going to commit to fixing before the next round of testing © 2001 Steve Krug © 2011 Steve Krug The debriefing Go around the room Everyone contributes three items from their list of nine problems Write on easel pad Leave some space for improvements/amendments People can say “Me too!” (add checkmarks) Treat all contributions with respect Not discussing yet Stick to observed problems! © 2001 Steve Krug © 2011 Steve Krug The debriefing Decide which are most serious Some magic happens here Voting/Dictatorship Not usually as hard as it seems BECAUSE THEY ALL SAW THE SAME BEHAVIOR Number them Copy the numbered list Ten is probably more than enough Leave space in between © 2001 Steve Krug © 2011 Steve Krug The debriefing Start at the top Work down the list Come up with rough idea of how you’ll fix them Who will do it The resources required When you’ve allocated the resources you can commit in the next month, STOP! Dramatic option: Tear off the rest of the list Crumple it up Throw it away © 2001 Steve Krug © 2011 Steve Krug © 2001 Steve Krug A great tip © 2001 Steve Krug And the companion volume… © 2001 Steve Krug © 2001 Steve Krug Thanks for all the fish Send any questions, feedback, gripes to skrug@sensible.com © 2001 Steve Krug © 2011 Steve Krug © 2011 Steve Krug