Ethnography in Designing SocioTechnical Environments
Wednesday, March 15, 2006
Stefan Carmien
Melissa Dawe
Anja Kintsch
Computer Scientists and People
?
?
Our formal training as technologists and designers
often leaves gaps in how to do research in the
socio side of socio-technical environments..
What we want to share
Our experiences in using ethnography as
a tool to understand the context and the
stakeholders in our work designing
systems for persons with cognitive
disabilities and support communities.
Ethnographic Methods
• What are they?
– Who uses them & why
• What are their attributes
• What are the skills of an ethnographer?
• What artifacts do they produce?
Qualitative Ethnography
• We are only focusing on Qualitative
aspects of ethnography, why?
–
–
–
–
–
Participant observation
Fieldnotes
Coding
Hypothesis generation
Semi-structured interviews
• Inspired by Grounded theory
– Allow topics to emerge from raw data
Ethnography and L3D
• How we can use it
• Why we must use it
Three Projects
• Stefan - MAPS
• Melissa – Mobile Communication Study
• Anja - Smart Care
Stefan
• MAPS system research focus
– How person with cognitive disabilities live & learn
– How new AT is used
• Roles:
– Person with cognitive disabilities
– Caregivers
•
•
•
•
Parent
Job coach
Transition team
Group home staff
– Study in pairs
Stefan - Methods
– Semi- structured interviews
• At end of usability studies
• Exit interviews at end of MAPS introduction
– Participant observation <typically 20 hours>
• Several days just hanging out
– Activities of daily life (ADL)
• Introduce MAPS
• Caregivers making scripts
• Three scripts
– Controlled environment (home)
– Less controlled environment
– Open environment (out of home)
Stefan Methods (cont’d)
– Each participant observation session was documented
• Fieldnotes (little note pad)
• Digital recorder
– Post Observation write-up
• Transfer Fieldnotes to log <annotated with observations>
• Roughly transcribe recording to log <ditto>
– One hour observation = 1.7 hours entry in log
– Each Dyad’s log was between 20 and 30 pages
• Coding & pattern extraction
• Illustrative quotations from recordings
• Each finished dyad informed the approach of the study of the
remaining ones
Stefan -
How I have used the data
• Experiencing the life of my population (both parts of the dyad)
– Daily variations in ability
– Details of what works and doesn't work in scripts (Voice, size of
steps, internal scripts)
– ‘Battle of time’ of caregivers
• Design with Scenarios (usability & editor features)
– Caregivers implicit requests
– Caregivers hopes, expectations, & fears
• Left to do:
– Code dyad studies
– Re-code Melissa's interviews
– Extract abstractions from coding
– Use quotes in dissertation
Melissa: Mobile Communication Study
Research Focus
1) How can technology support the mobile
communication needs of young adults with
moderate cognitive disabilities?
2) How can caregivers and individuals with
cognitive disabilities authentically participate
in the design process?
Melissa: Mobile Communication Study
Methods
Phase 1: Semi-structured interviews with families
– Recruiting participants
– Developing an interview guide
– Conducting interviews in peoples’ homes
– Coding
– Qualitative analysis
– Maintaining a relationship with participants
Melissa: Mobile Communication Study
Methods (cont’d)
• Phase 2: Interviews and observations focused on
mobile communication
• Phase 3: Technology “probe” study
– Work with families (participatory design) to design
UI for their child
– 8 week field trial to study co-evolution of users and
technology
– Technology-as-ethnographer through detailed
logging
– Design-in-use: requirements emerge through ‘doing’
Melissa: Mobile Communication Study
Participatory Design Activity
Melissa: Mobile Communication Study
working with AbleLink
• AbleLink “Pocket Ace”
software is a cell phone
application designed for
individuals with cognitive
disabilities
• In exchange for use of their
software, I will provide
usability feedback and
feature requirements
• I have full access to source
code, dev support
Anja: Observations on observing and
interviewing Imagine!
• “Fact-finding investigation of factors related to
caregiver activities, quality of life, and use of
personal technologies among individuals with
cognitive disabilities who are residents in group
homes and community settings.”
• Inform the development of context-aware
technologies.
• Face-to-face semi-structured interviews and in situ
observations
• Part of RERC
Goals of the Questionnaire
• Made sure our questions reflected our research questions.
• Interview procedures and questions intended to collect quantitative
as well as qualitative data
• Made questions similar for all respondents: managers, group home
caregivers, parents, and clients.
Experiences with the
Questionnaire
• Caregivers interpreted the questions differently than we
intended.
– We were looking for specific answers so we could say x out of y
say z. - that didn’t happen.
• “Please describe a situation where a client who you know well
can “almost” do something that’s important to them? …what
would be needed to allow them to do the task independently?”
- no answer involved A.T.
Timing challenges with the
Questionnaire
• Once we settled on the questionnaire
we were bound to it because we had to
get 58 interviews completed and we
were behind schedule.
Interviewing Caregivers
Different group home caregivers responded to my
presence differently:
–
–
–
–
–
Guardedness - are you a spy from management?
Defensiveness - I am good at my job!
You are here to fix things - let me tell you the problems
Open and confident
For some it was enjoyable for others it felt like a
requirement
• Was not capture in any formal method
Observing when people with
cognitive disabilities are present
• We planned on passive observation,
but I ended up being participatory
observer
– Observer effect compounded: Don’t
respect the boundaries of observer and
observed - would come up and interact
with me.
How to do research with people
with cognitive disabilities Potential Problems?
• Tangential thinking and lack of familiarity
with the context
• Caregivers answering or interpreting too
much
• When working with clients with limited
speech. How do you offer choices without
influencing the answer?
• Foils?
What are your most favorite things
to do each week?
Potentially
correct
choices as
offered by
Imagine!
Foils: are
offered by
Imagine! but
client not
participating in
these classes.
Potentially correct
choices of daily
activities but are
not Imagine!
classes or
necessarily
considered
selectable events
Incorrect and
absurd foils
What We Got Using Ethnographic Methods That
We Could Not Have Gotten Otherwise
• Stefan:
– Understanding caregivers & relationships
– Getting young adults ability & needs
• Vary over time
• Typical teenage issues
• uneven map of abilities (unexpected
faculties)
– Scenario based design
What We Got Using Ethnographic Methods That
We Could Not Have Gotten Otherwise
• Melissa:
– Understand that for families, assistive
technology includes kitchen timers, memo
recorders, Yahoo! games, etc.
– Learn the importance of mobile communication
in supporting independence, socialconnectedness, sense of safety for families
– A design process where families guide the
design of their own technology
What We Got Using Ethnographic Methods That
We Could Not Have Gotten Otherwise
• Anja:
– Compare what management sees as important
issues verses what caregivers on the ground see
as important
– See the extreme limits of many clients in the
group home environment - maybe it isn’t the
best place to work on independence
– See that group homes are like large families
Our goal
We hope to spark a discussion with the larger L3D
community that have used or are interested in
using these methods.
Resources- Classes
• Classes:
– ANTH 7300 -- Research methods in cultural anthropology
– COMM 6030/001 -- Interpretive Research Methods
– SOCY 5031/001 -- Research Design
Resources- Tools
• Digital recorders (Olympus)
• Transcribing software - Olympus DSS
player (adaptive speed control)
• Coding tools:
– HyperResearch
– NVIVO
Resources- Texts
• Texts:
– Analyzing and Interpreting Ethnographic Data, Lecompte
– Analyzing Social Settings, Lofland & Lofland
– Basics of Qualitative Research: Techniques and Procedures
for Developing Grounded Theory, Strauss & Corbin
– Contemporary Field Research, Emerson
– Qualitative Research Methods, Berg
– Research Methods in Anthropology, Bernard
– Writing Ethnographic Fieldnotes, Emerson, Fretz, Shaw
Resources- People
• Local Expertise:
–
–
–
–
–
Leysia Palen
Tammy Sumner
Sophia Liu
Heinrich Schwartz (Comm Dept.)
Kathleen Tierney (Sociology Dept.)
Answers to the questions on slide 4
•
•
What are they?
Ethnography is "the study of people in their own environment". So it's a collection of methods that are intended to understand something about
people, and involve some sort of fieldwork in their environment. Different methods are appropriate based on the setting, and the research
questions.
•
•
Who uses them and why?
Historically, first were anthropologists. The classic method was participant observation in an extended study, often studying foreign cultures (think
Margaret Mead living in Samoa). Now sociologists,communication studies researchers, HCI, even marketing departments, etc.,a wide array of
researchers interested in some aspect of human behavior in a natural setting.
•
-
What are the attributes?Studying people, some aspect of human behavior (including social behavior)
In the field -- it involves studying people in their environmentOften involve data collection that is analyzed qualitatively, but also conducive to quantitative analysis (or some hybrid)Often used for exploratory research or early phases, when research questions are being definedCan be used in complement with other research methods (e.g. formal surveys, etc.) in "data triangulation" (gaining a more complete view by
approaching the problem from different perspectives)Traditionally assumed the researcher did not alter the people or environment under study, but now there are types of ethnography where this is
definitely a desired outcome (action research, design research like we do!)
-
-
What are the skills of an ethnographer?Depends on techniques used (interviews, observations, extended fieldwork, etc.), but in general:
•
•
•
•
-
study designdoing the fieldworkqualitative analysis (usually coding and analyzing large amounts of data)writing (presentation of ethnographic research is usually texts, including articles, books, trend in 'novel-like' writing style)
What artifacts does it produce?•
•
I kind of answered this above, usually texts, often books. The main 'artifact' in traditional ethno is a theory or theories about human behavior.HCI has the unique quality of developing tools through ethnographic methods.