Ethnography in Designing SocioTechnical Environments Wednesday, March 15, 2006 Stefan Carmien Melissa Dawe Anja Kintsch Computer Scientists and People ? ? Our formal training as technologists and designers often leaves gaps in how to do research in the socio side of socio-technical environments.. What we want to share Our experiences in using ethnography as a tool to understand the context and the stakeholders in our work designing systems for persons with cognitive disabilities and support communities. Ethnographic Methods • What are they? – Who uses them & why • What are their attributes • What are the skills of an ethnographer? • What artifacts do they produce? Qualitative Ethnography • We are only focusing on Qualitative aspects of ethnography, why? – – – – – Participant observation Fieldnotes Coding Hypothesis generation Semi-structured interviews • Inspired by Grounded theory – Allow topics to emerge from raw data Ethnography and L3D • How we can use it • Why we must use it Three Projects • Stefan - MAPS • Melissa – Mobile Communication Study • Anja - Smart Care Stefan • MAPS system research focus – How person with cognitive disabilities live & learn – How new AT is used • Roles: – Person with cognitive disabilities – Caregivers • • • • Parent Job coach Transition team Group home staff – Study in pairs Stefan - Methods – Semi- structured interviews • At end of usability studies • Exit interviews at end of MAPS introduction – Participant observation <typically 20 hours> • Several days just hanging out – Activities of daily life (ADL) • Introduce MAPS • Caregivers making scripts • Three scripts – Controlled environment (home) – Less controlled environment – Open environment (out of home) Stefan Methods (cont’d) – Each participant observation session was documented • Fieldnotes (little note pad) • Digital recorder – Post Observation write-up • Transfer Fieldnotes to log <annotated with observations> • Roughly transcribe recording to log <ditto> – One hour observation = 1.7 hours entry in log – Each Dyad’s log was between 20 and 30 pages • Coding & pattern extraction • Illustrative quotations from recordings • Each finished dyad informed the approach of the study of the remaining ones Stefan - How I have used the data • Experiencing the life of my population (both parts of the dyad) – Daily variations in ability – Details of what works and doesn't work in scripts (Voice, size of steps, internal scripts) – ‘Battle of time’ of caregivers • Design with Scenarios (usability & editor features) – Caregivers implicit requests – Caregivers hopes, expectations, & fears • Left to do: – Code dyad studies – Re-code Melissa's interviews – Extract abstractions from coding – Use quotes in dissertation Melissa: Mobile Communication Study Research Focus 1) How can technology support the mobile communication needs of young adults with moderate cognitive disabilities? 2) How can caregivers and individuals with cognitive disabilities authentically participate in the design process? Melissa: Mobile Communication Study Methods Phase 1: Semi-structured interviews with families – Recruiting participants – Developing an interview guide – Conducting interviews in peoples’ homes – Coding – Qualitative analysis – Maintaining a relationship with participants Melissa: Mobile Communication Study Methods (cont’d) • Phase 2: Interviews and observations focused on mobile communication • Phase 3: Technology “probe” study – Work with families (participatory design) to design UI for their child – 8 week field trial to study co-evolution of users and technology – Technology-as-ethnographer through detailed logging – Design-in-use: requirements emerge through ‘doing’ Melissa: Mobile Communication Study Participatory Design Activity Melissa: Mobile Communication Study working with AbleLink • AbleLink “Pocket Ace” software is a cell phone application designed for individuals with cognitive disabilities • In exchange for use of their software, I will provide usability feedback and feature requirements • I have full access to source code, dev support Anja: Observations on observing and interviewing Imagine! • “Fact-finding investigation of factors related to caregiver activities, quality of life, and use of personal technologies among individuals with cognitive disabilities who are residents in group homes and community settings.” • Inform the development of context-aware technologies. • Face-to-face semi-structured interviews and in situ observations • Part of RERC Goals of the Questionnaire • Made sure our questions reflected our research questions. • Interview procedures and questions intended to collect quantitative as well as qualitative data • Made questions similar for all respondents: managers, group home caregivers, parents, and clients. Experiences with the Questionnaire • Caregivers interpreted the questions differently than we intended. – We were looking for specific answers so we could say x out of y say z. - that didn’t happen. • “Please describe a situation where a client who you know well can “almost” do something that’s important to them? …what would be needed to allow them to do the task independently?” - no answer involved A.T. Timing challenges with the Questionnaire • Once we settled on the questionnaire we were bound to it because we had to get 58 interviews completed and we were behind schedule. Interviewing Caregivers Different group home caregivers responded to my presence differently: – – – – – Guardedness - are you a spy from management? Defensiveness - I am good at my job! You are here to fix things - let me tell you the problems Open and confident For some it was enjoyable for others it felt like a requirement • Was not capture in any formal method Observing when people with cognitive disabilities are present • We planned on passive observation, but I ended up being participatory observer – Observer effect compounded: Don’t respect the boundaries of observer and observed - would come up and interact with me. How to do research with people with cognitive disabilities Potential Problems? • Tangential thinking and lack of familiarity with the context • Caregivers answering or interpreting too much • When working with clients with limited speech. How do you offer choices without influencing the answer? • Foils? What are your most favorite things to do each week? Potentially correct choices as offered by Imagine! Foils: are offered by Imagine! but client not participating in these classes. Potentially correct choices of daily activities but are not Imagine! classes or necessarily considered selectable events Incorrect and absurd foils What We Got Using Ethnographic Methods That We Could Not Have Gotten Otherwise • Stefan: – Understanding caregivers & relationships – Getting young adults ability & needs • Vary over time • Typical teenage issues • uneven map of abilities (unexpected faculties) – Scenario based design What We Got Using Ethnographic Methods That We Could Not Have Gotten Otherwise • Melissa: – Understand that for families, assistive technology includes kitchen timers, memo recorders, Yahoo! games, etc. – Learn the importance of mobile communication in supporting independence, socialconnectedness, sense of safety for families – A design process where families guide the design of their own technology What We Got Using Ethnographic Methods That We Could Not Have Gotten Otherwise • Anja: – Compare what management sees as important issues verses what caregivers on the ground see as important – See the extreme limits of many clients in the group home environment - maybe it isn’t the best place to work on independence – See that group homes are like large families Our goal We hope to spark a discussion with the larger L3D community that have used or are interested in using these methods. Resources- Classes • Classes: – ANTH 7300 -- Research methods in cultural anthropology – COMM 6030/001 -- Interpretive Research Methods – SOCY 5031/001 -- Research Design Resources- Tools • Digital recorders (Olympus) • Transcribing software - Olympus DSS player (adaptive speed control) • Coding tools: – HyperResearch – NVIVO Resources- Texts • Texts: – Analyzing and Interpreting Ethnographic Data, Lecompte – Analyzing Social Settings, Lofland & Lofland – Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, Strauss & Corbin – Contemporary Field Research, Emerson – Qualitative Research Methods, Berg – Research Methods in Anthropology, Bernard – Writing Ethnographic Fieldnotes, Emerson, Fretz, Shaw Resources- People • Local Expertise: – – – – – Leysia Palen Tammy Sumner Sophia Liu Heinrich Schwartz (Comm Dept.) Kathleen Tierney (Sociology Dept.) Answers to the questions on slide 4 • • What are they? Ethnography is "the study of people in their own environment". So it's a collection of methods that are intended to understand something about people, and involve some sort of fieldwork in their environment. Different methods are appropriate based on the setting, and the research questions. • • Who uses them and why? Historically, first were anthropologists. The classic method was participant observation in an extended study, often studying foreign cultures (think Margaret Mead living in Samoa). Now sociologists,communication studies researchers, HCI, even marketing departments, etc.,a wide array of researchers interested in some aspect of human behavior in a natural setting. • - What are the attributes?Studying people, some aspect of human behavior (including social behavior) In the field -- it involves studying people in their environmentOften involve data collection that is analyzed qualitatively, but also conducive to quantitative analysis (or some hybrid)Often used for exploratory research or early phases, when research questions are being definedCan be used in complement with other research methods (e.g. formal surveys, etc.) in "data triangulation" (gaining a more complete view by approaching the problem from different perspectives)Traditionally assumed the researcher did not alter the people or environment under study, but now there are types of ethnography where this is definitely a desired outcome (action research, design research like we do!) - - What are the skills of an ethnographer?Depends on techniques used (interviews, observations, extended fieldwork, etc.), but in general: • • • • - study designdoing the fieldworkqualitative analysis (usually coding and analyzing large amounts of data)writing (presentation of ethnographic research is usually texts, including articles, books, trend in 'novel-like' writing style) What artifacts does it produce?• • I kind of answered this above, usually texts, often books. The main 'artifact' in traditional ethno is a theory or theories about human behavior.HCI has the unique quality of developing tools through ethnographic methods.