Games User
Research
{
Cyril Rebetez
User Research Team
Sony Computer Entertainment Europe
User Research at Sony PlayStation
1. Who are Sony Computer Entertainment?
2. Games user research vs usability testing
3. How to do games user research
Recruitment
Methods
Analysis
Reporting
(Break)
Activities – setup a games user research
Who are Sony Computer
Entertainment?
Who are Sony Computer
Entertainment?
What does the User Research
Team do?
Usability and User Research – During Development
…
Now is a good time to watch a video
What does the User Research
Team do?
What does the User Research
Team do?
Research Projects
What does the User Research
Team do?
Games user research
vs
Usability?
effectiveness
efficiency
usability
learnability
memorability
satisfaction
Productivity
effectiveness
efficiency
G.U.R
satisfaction
Engagement
Not designed for efficiency
Games must challenge
(in the right way)
Passionate Audience
Different interaction methods
Different interaction methods
Different interaction methods
usability G.U.R

Reduce Hassle

Reduce interface Hassle

Maximise productivity

Maximise engagement

Consistent simplicity

Progressive complexity

Understanding

Understanding & performing

Focus on effectiveness & efficiency

Focus on fun & emotion

Mainly behaviour & some attitude

Both behaviour & attitude
Researching
User Experience
How to do it?
How to do it?
1.
2.
3.
4.
5.
6.
7.
Kick off – What are we looking for?
Preparation – How do we do it?
Recruitment – Who should do it?
Moderation – How to make them do it?
Observation – What to look for?
Analysis – What does it mean?
Reporting – How to explain it?
1. Kick off
Objectives
User Research kick-off
• Meet with the people who make the game,
• What do they want to know?
• Everything
• Define the objectives
• Who are the audience?
• Decide how we can answer their questions.
2. Methods
1 to 1 testing
1 to 1 testing
Deep testing with 1 user at a time
Focus on observation and interviews
1 to 1 testing
What questions can it answer?
• Anything based around understanding, including:
• Do players know what to do?
• Do players know where to go?
• Do players know how to use this weapon or ability?
• What do players think about a feature?
• …
1 to 1 testing
“Make sure the VITA can see the cards”
1 to 1 testing
“Make sure the VITA can see the cards”
1 to 1 testing
1 to 1 testing
Why is it suitable?
• Allows close observation of the issues
• Can ask questions to the players
Multi-seat testing
Multi-seat testing
10 users at a time
Focus on quantitative data
Multi-seat testing
What questions can it answer?
•
•
•
•
Are there any difficulty spikes?
Can players complete the game?
How long does it take players to learn all of the mechanics?
And more…
RESOGUN
Multi-seat testing
Multi-seat testing
Why is it suitable?
• Many participants increase reliability
• Questionnaires can get quantitative data
• Can test longer sections of gameplay
Home Testing
Real world setting
Uncover issues you can’t find in a lab
Home Testing
Home Testing
?
Home Testing
?
Home Testing
Diary Studies
Real world setting
Uncover issues that reveal themselves over time
Diary Studies
Diary Studies
3. Recruitment
Audience
Diversity
Diversity
 More than 34% of gamers are 18+ females
 Less than 18% of gamers are under 17 males
 25% of the gamers are 50+ years old
 Average age of gamers is 35 years old
Defining users profiles
o
Use market research outcomes
o
Define with stakeholders
o
o
o
o
What is the vision?
Who is likely to play / purchase / use the product?
Is it an established franchise?
Trying to break into a new market?
How many users?
Usability :
“5 users will typically uncover 80% of the usability issues” that the
whole target audience will experience (Nielsen, 1998)
Recruitment
Group profile definition
 Demographics (age, gender)
 Gaming profile: Hardcore, Core, Casual, Fanatic, Kids, etc…)
 Type(s) of games played
 Hours of play per week (online & offline)
 Number of games purchased in the last 3 months
 Games & system(s) owned (e.g. 360° + Kinect)
 Other criteria (e.g. franchise awareness, play preferences, etc.)
Recruitment
Example of screener
criterion
requirement
screener
Age range(s)
5x teens 13 to 16 (equal spread)
Discard: <13 & >16 yrs old
Gender(s)
3x boys & 2x girls
Gaming habit
2 to 5 hrs/week offline & 0 online
Discard <2 & >5 hrs/week
purchase
Max 2 game in last 3 months
Discard >2 games in 3months
System own.
Own PS3 but no 360
Discard no-PS3 and own-360
Other
Most have finished LBP
Discard non-LBP completers
4. Moderation
Moderation = Craft
Some tips






Keep participants comfortable
Ask non-leading and open questions
Give users the voice but lead them where you want
Resist the urge to help
Stay objective, impartial and relaxed
If you make a mistake, carry on
5. Observation
Note taking
Outsource!
moderator
Observer/ Note-taker
Note-taking
Tips
o
Write down in advance what you want to observe
o
o
Tally each time a user experiences the issue, follow up
Focus on what happens and why, no just what participants say
6. Analysis
analysing data
Gather
ather
Aggregate
ggregate
P
rioritise
Sort
Combining data
Secret recipe
7. Reporting
Workshop solutions
Ensure your research has the maximum impact
Write a full report
The final report
A document to refer to
A list of issues to follow up on
Re-testing
Test
Did it
work?
Find Issues
Team
introduces
fixes
• Games User Research Special Interest Group:
http://www.linkedin.com/groups?mostPopular=&gid=1873014
• Games User Research Summit:
http://emiie.iat.sfu.ca/gur/
• Game User Research Library:
http://sites.google.com/site/gamesuserresearch/
Part 2 - Activity
Activity
5 groups
Find a game you know
1.
2.
o
4.
5.
o
7.
30 min
Discuss profile(s) of users
Kick off – talk to the producer (i.e. me)
Setup a research plan
3.
6.
Check on your phone / browser history
Method, moderation guide, questionnaires…
Write a proposal on a poster (pitch)
Present / discuss
30 min
30 min
Thank You!
Our Mailing List
-
Helping with research
Job opportunities
Training
 in central London…
Questions?
Email: cyril_rebetez@scee.net
http://tinyurl.com/SonyUserResearch