Chapter 8
Data analysis, interpretation, and
presentation
Outline





Qualitative and quantitative
Simple quantitative analysis
Simple qualitative analysis
Using theoretical frameworks
Presenting the findings
Qualitative and quantitative

Quantitative data is data that is in the form of
numbers, or that can easily be translated into
numbers

Examples: number of years of work
experience, number of projects, number of
minutes to carry out a task
Qualitative and quantitative

Qualitative data is data that is difficult to
measure, count, or express in numerical
terms that make sense
Qualitative and quantitative


All the forms of data gathering discussed in
previous chapter may result in qualitative and
quantitative data
Examples:

on a questionnaire


participants’ age or number of software packages
used are quantitative data
comments are qualitative data
Data analysis

First steps in analyzing data

Interviews


Questionnaires


data transcription, finding categories or patterns of
response
data cleansing, data filtering (subsets of data)
Observations

Notes write-up, data transcription, synchronization
between different data recordings
Data analysis

Simple quantitative analysis

Averages





Mean (ค่าเฉลี่ย)
Median (มัธยฐาน)
Mode (ฐานนิยม)
Percentages
Data collation (Excel)


Finding out outliers
Finding out patterns (graphical representation)
Simple qualitative analysis



Unstructured - are not directed by a script.
Rich but not replicable.
Structured - are tightly scripted, often like a
questionnaire. Replicable but may lack
richness.
Semi-structured - guided by a script but
interesting issues can be explored in more
depth. Can provide a good balance between
richness and replicability.
From: www.id-book.com
Data analysis

Simple qualitative analysis



Identifying recurring patterns or themes
Categorizing data
Analyzing critical incidents
Data analysis

Simple qualitative analysis

Identifying recurring patterns or themes



Study goals provide an orienting focus for the
formulation of themes
A balance between generalness and specificity
Relate to various aspects, e.g., to behavior, to user
group, to places and situations
Data analysis

Simple qualitative analysis

Categorizing data



The data is divided up into elements and each
element is then categorized
The categorization scheme may arise from the data
itself, or it might originate elsewhere in a wellrecognized categorization scheme, or a combination
of these two approaches
The goal of the study largely determines which
categories to use
Data analysis

Simple qualitative analysis

Categorizing data

The challenging aspects are



1) determining meaningful categories that are not overlap
each other in any way
2) deciding on the appropriate granularity for the categories,
e.g. at word, phrase, sentence, or paragraph level
The categorization scheme used must be reliable ->
inter-rater reliability
Data analysis

Simple qualitative analysis

Categorizing data


Content analysis involves categorizing the data and
then studying the frequency of category occurrences
Discourse analysis focuses on the dialog, i.e. the
meaning of what is said, and how words are used to
convey meaning

Conversation analysis focuses on how conversation is
conducted, i.e. how conversation starts, how turn-taking is
structured, and other rules of conversations
Data analysis

Simple qualitative analysis

Analyzing critical incidents

Two basic principles


1) reporting facts regarding behavior is preferable to the
collection of interpretations, ratings, and opinions based on
general impressions -> well-planned observation sessions
2) reporting should be limited to those behaviors which,
according to competent observers, make a significant
contributions to the activity in either desirable or an
undesirable way
Using theoretical frameworks



Grounded theory
Distributed cognition
Activity theory
Grounded theory



The theory derived is grounded in the data
The aim is to develop a theory that fits a set
of collected data
In this context, theory means ‘a set of welldeveloped concepts related through
statements of relationship, which together
constitute an integrated framework that can
be used to explain or predict phenomena’
(Strauss & Corbin, 1998)
Grounded theory



Development of a ‘grounded’ theory
progresses through alternating data collection
and data analysis
Data gathering is driven by the emerging
theory
This approach continues until no new insights
emerge and the theory is well-developed
Grounded theory

Researchers need to maintain a balance
between objectivity and sensitivity

Objectivity is needed to maintain accurate
and impartial interpretation of events

Sensitivity is required to notice the subtleties
in the data and identify relationships between
concepts
Grounded theory

Main idea is to identify and define the
properties and dimensions of relevant
categories and then to use these as the basis
for constructing a theory

Category identification and definition is
achieved by ‘coding’ the data
Grounded Theory

Three levels of ‘coding’

Open: identify categories

Axial: flesh out and link to subcategories

Selective: form theoretical scheme
Grounded Theory




Questioning
Analysis of a word, phrase, or sentence
Comparisons
Researchers are encouraged to draw on own
theoretical backgrounds to inform analysis
Distributed Cognition

The people, environment & artifacts are
regarded as one cognitive system

Used for analyzing collaborative work

Focuses on information propagation &
transformation
From: www.id-book.com
Distributed Cognition

Results in an event-driven description which
emphasizes information and its propagation
through the cognitive system under study

The granularity of analysis varies depending
on the activities and cognitive system being
observed and the research or design
questions being asked
Distributed Cognition

Ed Hutchins emphasizes that an important
part of doing a distributed cognition analysis
is to have a deep understanding of the work
domain that is being studied

There is no single way of doing a distributed
cognition analysis
Distributed Cognition



A good way to begin analyzing and
interpreting the data collected is to describe
the official work practices
Any breakdowns, incidents, or unusual
happenings should be highlighted
While writing these observations down, it is
good to start posing specific research
questions related to them and contemplate
further
Distributed Cognition

Problems can be described in terms of the
communication pathways that are being
hindered or the breakdowns arising due to
information not propagating effectively from
one representational state to another
Distributed Cognition

It can show when different technologies and
the representations displayed via them are
effective at mediating certain work activities
and how well they are coordinated

Concepts:



Cognitive system
Communicative pathways
Propagation of representational states
Activity Theory



Explains human behavior in terms of our
practical activity with the world
Provides a framework that focuses analysis
around the concept of an ‘activity’ and helps
to identify tensions between the different
elements of the system
Two key models: one outlines what
constitutes an ‘activity’; one models the
mediating role of artifacts
Individual model
From: www.id-book.com
Individual model



Operations – routinized behaviors that
require little conscious attention, e.g. rapid
typing
Actions – behavior that is characterized by
conscious planning, e.g. producing a glossary
Activity – provides a minimum meaningful
context for understanding the individual
actions, e.g. writing an essay
Individual model

There may be many different operations
capable of fulfilling an action

Many actions capable of serving the same
activity
Individual model

There is an intimate and fluid link between
levels

Actions can become operations as they
become more automatic

Operations can become actions when an
operation encounters an obstacle, thus
requiring conscious planning
Individual model

If ‘motive’ changes then an activity can
become an action

Activities relate to others while actions may
be part of different activities
The role of artifacts

Artifacts can be physical, e.g.



A book
A stone
Artifacts can be abstract, e.g.


A system of symbols
A set of rules
Engeström’s (1999) activity
system model
From: www.id-book.com
Presenting the findings

Only make claims that your data can support

The best way to present your findings depends on
the audience, the purpose, and the data gathering
and analysis undertaken

Graphical representations may be appropriate for
presentation
From: www.id-book.com
Presenting the findings

Other techniques are:

Rigorous (clear syntax and semantics) notations,
e.g. UML

Using stories, e.g. to create scenarios

Summarizing the findings
From: www.id-book.com
Summary

The data analysis that can be done depends
on the data gathering that was done

Qualitative and quantitative data may be
gathered from any of the three main data
gathering approaches

Percentages and averages are commonly
used in Interaction Design
Summary



Mean, median and mode are different kinds
of ‘average’ and can have very different
answers for the same set of data
Grounded Theory, Distributed Cognition and
Activity Theory are theoretical frameworks to
support data analysis
Presentation of the findings should not
overstate the evidence
Chapter Review
Chapter 1-7
Chapter 1




What is interaction design?
Interaction Designer vs. Software Engineers
User experience
Usability goals






Effectiveness (ประสิ ทธิ ผล)
Efficiency (ประสิ ทธิ ภาพ)
Safety
Utility
Learnability
Memorability
Chapter 1


Usability Criteria
Norman’s Design Principles





Visibility
Feedback
Constraints
Consistency
Affordances
Chapter 2

Understanding the problem space




Are there problems with an existing product or
user experience? If so, what are they?
Why do you think there are problems?
How do you think your proposed design ideas
might overcome these?
If you have not identified any problems and
instead are designing for a new user experience,
how do you think your proposed design ideas
support, change, or extend current ways of doing
things?
Chapter 2

Conceptual model




Metaphors and analogies
Concepts: task-domain objects, objects’
attributes, operations performed on objects
Relationships between concepts
Mappings between concepts and task-domain the
system is designed to support
Chapter 2


Interface metaphors
Interaction Types




Instructing
Conversing
Manipulating
Exploring
Chapter 3

Users’ cognition






Attention
Perception and recognition
Memory
Learning
Reading, speaking, and listening
Problem-solving, planning, reasoning, decisionmaking
Chapter 3

Cognitive frameworks





Mental models
Theory of action
Information processing
External cognition
Distributed cognition
Chapter 4

Social mechanisms in communication and
collaboration



Conversational mechanisms
Coordination mechanisms
Awareness mechanisms
Chapter 5

Affective aspects

User frustration



Sources of user frustration, e.g. error messages,
appearance
Persuasive technologies
Anthropomorphism in interaction design, i.e.
virtual agents
Chapter 6

Interface types






Command interfaces
WIMP/GUI interfaces
Advanced graphical interfaces
Web-based interfaces
Speech interfaces
Mobile interfaces
Chapter 7

Data gathering



Interviews
Questionnaires
Observation