Iterative Analysis: Primary and Secondary Coding (Tracy, 2013) - Cycle between finding patterns in the data and connecting to existing theory/ literature - Steps 1. Prepare the raw data 2. Primary Cycle - Immerse yourself in the data - Create “open” codes 3. Create a codebook 4. Secondary Cycle - Create hierarchical category’s 5. Revisit the scholarly literature on your topic to focus rest of analysis 6. Create relationships between category’s - Conceptual model Preparing Raw Data - Assign pseudonyms to names of specific people in your data - Organize the data in the order you plan to analyze : Chronologically : By type (interview vs. observation) : By source (employee vs. manager / case1 vs. case 2 / key informants vs. other informants) Primary Cycle - Inductive 1. Data Immersion - Read through all of your data several times : What’s happening here? What strikes you as interesting? - Discuss in your team - Compare your reflections with others : Do you see any stories or themes emerging in your data? 2. Open Coding - Identifying data as belonging to or representing some type of phenomenon - Codes are words and short phrases (label) that captures that salient attribute of your data - Stay descriptive and from the data itself : the goal is to detail “who, what, where” (not why or how) - Example “Open” coding How to “OPEN” Coding - Unit of coding: sentence-paragraph - Creating code: stick closely to what the participants of your data said/ what you saw in your observations- in vivo codes - Constant Comparative Method: compare new uncoded data with your old codes- try to apply the same code if it “fits” to new data. - Simply describe- do not yet interpret!: want to capture participants meanings Open codes are the raw ingredients for your eventual interpretation Tips - - Start with your best interview/observations so far Do NOT code data looking only for answers to your sub-questions : Point of open coding is to capture the participants meanings, not necessarily looking for an answer to your RQ yet! You can change/refine your codes as you go along Open codes may range from 20 to 300 (or more) Creating a Codebook - Codebook is a map to help you navigate through your coded data 1. Clean up codes - Start by creating a list of all your codes (teams combined coding efforts) - Having a meeting to discuss the codes and organize them : Immediately combine codes that are the exact same : Examine codes that seem to be similar- through discussion determine if they are in fact the same or not 2. Code instance & RQ - After cleaning up for duplicate codes, count how many times each code appears in your data set : If a code appears only once or very infrequently, you have a decision to make (decide it is unimportant or try to collect more data) - Assess your codes against your initial research question : Does it look like you have answered your RQ? : If not, collect more data; potentially revise your RQ 3. Definitions & Quotes - Create definitions for all remaining codes : Should explain exactly what the code means in the context of your data : Good to discuss definitions as a team - Choose an exemplar quote in the data that best represents each code Secondary Cycle Coding – Deductive - Organize, synthesize, and categorize your open codes into interpretive concepts - Tracy calls them second-level codes, I prefer the term “hierarchical category” - Category is also a code- but it is like an umbrella that encompasses several codes that are similar in some way : “lumping” coded data together - First level open codes are generated by the data, second level categories are generated by the researcher. Why? - A form of data reduction - First step towards noticing broader patterns in your data - Should begin to answer your research questions: moves from description to explain and theorize - You can begin to see where your analysis can “fill gaps” in the existing research, showing theoretical relevance Computer-Supported Qualitative Data Analysis - The software doesn’t ‘Build Theory’ – we do! “Qualitative coding is NOT the same as quantitative coding. The term itself provides a case in point in which the language may obscure meaning and method. Quantitative coding requires preconceived, logically deduced codes into which the data are placed. Qualitative coding, in contrast, means creating categories from interpretation of the data. Rather than relying in preconceived categories and standardized procedures qualitative coding has its own distinctive structure, logic and purpose.” – Charmaz, 1983 There are 2 main ways in which computers can help ‘Automating’ the (very old) processes of 1) Indexing 2) Cross-referencing Remember that both of these techniques for dealing with textual data have been around for a VERY long time! Indexing – Kelle, 1997 - An electronic index is usually constructed by storing index words together with the ‘addresses’ of text passages. - Such an address may contain the beginning and the end, in terms of line numbers, of a certain text passage to which the index word refers - Software programs which are based on these principles have been called ‘code-andretrieve’ programs (Kelle, 1995) - ‘Analogue’ example : Most common example of an index is found in the back of a book, either for subjects or names - ‘ATLAS.TI’ example : The ‘query’ function in the desktop version of ATLAS.TI lets you retrieve sections of the text that are associated with codes you have made : This means you can gather (and compare) all sections of your data (quotations) that you have associated with a particular code Cross-Referencing – Kelle, 1997 - Electronic cross references can be constructed with the help of so-called ‘hyperlinks’. By pressing a ‘button’ the user of a textual database can jump between the text passages which are linked together. - With the advent of hypertext and hypermedia technology it has often been forgotten that their main underlying principles have been widely known and applied for hundreds of years. - ‘Analogue’ example : A traditional example of a cross-reference that underpins a ‘hyperlink’ is in the margins of biblical texts – the ‘addresses’ in the margins ‘link’ to other relevant parts of the text - ‘ATLAS.TI’ example : ATLAS.TI desktop version comes with a set of standard hyperlink relations but the researcher can create new ones as needed, preferably guided by a theoretical framework. The standard relations are the following Continued by, Contradicts, Criticizes, Discusses, Expands, Explains, Justifies, Supports - Consider how the activity of creating hyperlinks can support your research/theorizing The Role of Qualitative Data Analysis Software Contrary to a quantitative analysis technique (like ‘logistic regression’ or ‘cluster analysis’) none of the following 3 steps can be conducted with an algorithm alone. At each step the role of the computer remains restricted to an intelligent archiving (‘code-and-retrieve’) system, and the analysis itself is always done by a human interpreter. – Kelle, 1997 1) Step one is the structuring of the material with the help of common-sense concepts or abstract theoretical concepts. Thereby, the code scheme can be developed before the coding takes place (‘axial coding’, ‘selective coding’) or while the material is being coded (‘open coding’, whereby ‘in vivo’ codes may be used). 2) First level coding is the necessary prerequisite for step two: a systematic comparison of text passages: text segments are retrieved and analyzed in order to discover ‘dimension’ which can be used as a basis for comparing different cases. 3) It is this comparison which becomes the basis of step three: the construction of concepts, types, and categories that form the building blocks of an emerging theory. Eventual contribution - Ultimately the end product should constitute some form of theory (in-the-making) - In the form of propositions, a conceptual model or other forms of (more or less) generalized/abstracted and broader conclusions or lessons - “An abstracted set of ideas and concepts with broader bearing on how to make sense of similar phenomenon in other settings” – Alvesson & Karreman, 2007 Ethics in Qualitative Research First, an often overlooked distinction – Roberts, 2015 Procedural Ethics Practical Ethics - The formal process of applying to and - Also known as process ethics, getting ethics approval from a situated ethics (Calvey, 2008) and research ethics committee (or embedded ethics (Whiteman, 2012) perhaps organization) before - The consideration given to ethics conducting a study throughout the research process as - Requires researchers to reflect on events or issues arise (Guillemin & their proposed methodology and Gillam, 2004) possible risks/harm to participants - The way in which researchers identify and others prior to the and respond to unforeseen ‘ethically commencement of research. important moments’ (G&G, 2004) throughout their study. Why do ethics matter? - Qualitative researchers interact with their participants : Relationships may form : Feeling of obligation to protect : Privacy and confidentiality - Qualitative research produces a great deal of context information : Ex: single case : Can inadvertently reveal identity of participants - Our analysis can impact people’s live : New workplace policies : Layoffs : Reputation An Alternative Perspective Interviewing Corporate Ethics – Welch et al., 2002 ‘Another researcher was often given very detailed advice on how to interpret the data being collected and what conclusions to draw: “I think you might have to change the focus of your thesis a bit because I don’t think the links between (Company X) and (Company Y) were as strong as perhaps you’d imagined.” In all but four cases, there was a seniority gap of at least ten years between the elites interviewed for our projects and the researcher. Some elites exploited this asymmetry, expressing impatience at having to waste time answering what they deemed to be obvious or irrelevant questions. However, others responded to the seniority gap by adopting a paternal attitude of instructing the less experienced researcher. At one point in the research, an interviewee articulated this role by exclaiming: “I feel like your father!”. The impact of the seniority gap is therefore an ambivalent one: on the one hand, researchers may be patronised and their comments overridden; on the other hand, elite individuals may take the time to inform and “enlighten” their junior instructor.’ Nine Ethics Principles 1. Do No Harm - Your study should not hurt anyone physically or psychologically - Principle stems from lessons learned from early research like the 1971 Stanford Prison Experiment (교도관과 수감자역으로 학생을 나눠서 실험했던 것. Research was called off) - Bottom Line: If your study seems to be hurting anyone, stop the study, even if this is problematic for your research design/plans. 2. Privacy and Anonymity - Two kinds of privacy: institutional and individual - Any group or organization participating in a research study has a reasonable expectation that its identity will not be revealed. - How will you conceal the identity of the individuals and institutions (orgs) you study? - No identifying information about the institution or individual should be revealed in written or other communication. - Consider how to anonymize descriptions, images, sketches, etc. Use pseudonyms (e.g. Company X) - Your participants may want their identities revealed. They may want to be acknowledged in your written product. Perhaps they see it as their “15 minutes of fame.” This raises other problems. - Bottom Line: Remove identifying information from your records. Seek permission from the participants of you wish to make public information that might reveal who they are or who the organization is. Use caution in publishing long verbatim quotes, especially if they are maging? to the organization or people in it. Often, these quotes can be located on the internet and traced to the speaker or author. 3. Confidentiality - Research is based in trust. - During your research, you might learn a considerable amount of personal information about your participants. - Your participant is entitled to expect that information they provide in your research will not be passed on to anyone else (e.g., their colleague, superiors, your friends, etc.) - When discussing research data with your research team, do so with sensitivity (e.g., respect your participants) - As a researcher, you are in a situation that you control. If you sense an interview might be moving in a very personal direction, you might have to stop the interview and suggest to the participant that they talk to a counsellor or other trusted support person. - Bottom Line: It is your responsibility to keep the information you learn confidential. Take care to not reveal details of one interviewee to another, e.g., “Jan said it’s pretty boring to work here, what do you think?” 4. Informed Consent - Individuals should be informed of the nature of the study and may choose whether or not to participate. - They should not be coerced into participation e.g., if a study is to be done in an organization, individuals within that group (e.g., students, workers) might feel that they cannot refuse when asked. There might be pressure placed on them by peers or by superiors - Consider the capacity of vulnerable groups to really give informed consent - Your research might diverge in a direction that causes participants to become uncomfortable or unwilling to continue, so consent people give in advance may not really be “informed”. - Bottom Line: your responsibility is to make sure that participants are informed, to the extent possible, about the nature of your study. If participants decided to withdraw from the study, they should not feel penalized for doing so – e.g., they should not be made to feel that their relationship with your or the VU is under threat if they decide not to take part/withdraw from the study. 5. Rapport and Friendship - Once participants agree to be part of a study, the researcher developed rapport in order to get them to disclose information. - Duncombe and Jessop (2005) bring out issues related to what they call faking friendship. - They suggest that the interviewer might put herself in the position of being a friend so as to get participants to disclose more information than they really want to. What is the issue here? - Bottom Line: Researchers should make sure that they provide an environment that is trustworthy. At the same time, they need to be sensitive to the power that they hold over participants. Researchers need to avoid setting up a situation in which participants think they are friends with the researcher. 6. Intrusiveness - Your study should not be intrusive for participants: intrusiveness can mean intruding on their time, intruding on their space, and intruding on their personal lives. - Make a reasonable estimate of the amount of time participation will take - Participants may not want you in their homes or classrooms – you might have to negotiate a neutral location for a discussion - Bottom Line: Experience and caution are the watchwords. Be sensitive to your research participants’ reactions and try to do what makes them comfortable. You are not an investigative journalist. Additional Observational Technique: Shadowing - Let the shadowee suggest a good time for you to shadow them : Try to get an overview of their routine ahead of time - Know what you are allowed to bring (security measures, etc) : Bring notebook and several pens in easy to carry bag : Tape recorder for impromptu interviews : Snacks to eat on the go : Dress in accordance with the norms of the organization - Follow the rules of the setting - Make shorthand notes to type up later; can also talk your reflections into a recording 7. Inappropriate Behavior - Individuals participating in a research study have a reasonable expectation that the researcher will not engage in conduct of a personal or sexual nature. - Here, researchers might find themselves getting too close to the participants and blurring boundaries between themselves and others. - We probably all know what we mean by inappropriate behavior. We know it should be avoided. Yet, there are documented examples of inappropriate behaviors between teachers and their minor students, between therapists and their patients, and between researchers and their participants. - Bottom Line: If you think you are getting too close to those you are studying, you probably are. Back off and remember that you are a researcher and bound by your code of conduct to treat those you study with respect. Also, speak up if you feel uncomfortable about your own situation. 8. Data Interpretation - A researcher is expected to analyze data in a manner that avoids misstatements, misinterpretations, or fraudulent analysis. - Use your data to fairly represent what you see and hear. Of course, your own lens will influence you. I am not suggesting that you strive for an objective stance. - Rather, I am pointing out the potential pitfalls of overinterpreting or misinterpreting the data you collect to present a picture that is not supported by data and evidence. - Bottom Line: You have a responsibility to interpret your data and present evidence so that others can decide to what extent your interpretation is believable. 9. Data Ownership and Rewards - In general, the researcher owns the work generated. - - This can be different if you have signed particular agreements with a company, especially if they have funded the research and have a claim to I.P. or data. Some researchers choose to archive data and make them available through databanks. Questions have been raised as to who actually owns such data. Some have questioned whether the participants should share in the financial rewards of publishing. Several ethnographers have shared a portion of their royalties with participants Bottom Line: In fact, most researchers do not benefit financially from their writing. It is rare that your work will turn into a bestseller or even be published outside your university. But if you have a winner on hand, you might think about sharing some of the financial benefits with others. Ethics in Online Research and General Tips Observation in online communities Observing the fora and pages where members of an online community interact and share information has increasingly been used as a method in qualitative research. This kind of research holds a lot of potential but also ethical risks. Why does this term have a negative connection? “Lurking” observation without participation on social media, websites, forums/discussion boards, and listservs. Types of Researcher Involvement Eysenbach and Till’s (2001) distinguish between the following roles: - ‘Passive’ use of existing data without researcher involvement in the online community - ‘Active’ participation by the researcher in the online community - ‘Traditional’ where data is generated through interviews or focus groups conducted online “While these categories continue to characterize much qualitative research online, increasingly hybrid approached are being adopted and research expanded into new types of internet communities, further increasing the complexity of ethical issues.” Issues with Using Online Data - Privacy, anonymity and confidentiality : Using quotes from a public internet forum means the words are searchable – even if you select one quote and change the person’s name, anyone can google the words and find your source and connect the quote with everything else they have said. Just because it is said in a ‘public’ forum, does not mean the participants see the forum as a ‘public space’ – Roberts, 2015 - Informed consent : Have you revealed your presence (see lurking)? Have you revealed your research purpose? Can users/members of the online community really decide to opt out of the research? How do you deal with new members who join? - Rapport and friendship : Sometimes you are already a part of the community, or you act as if you are. This can lead to a level of friendliness that causes members to share private information. Is it ethical to use friendship to encourage this disclosure? Preserving Anonymity and Pseudonymity in Online Data - Malik and Coulson (2013), in a qualitative study of permanent, involuntary childlessness, did not identify the name of the online community or the website address. - Quotations were anonymized and paraphrased and checked using search engines to ensure they were not traceable. - Hewitt- Taylor and Bond (2012) provided pseudonyms, didn’t name websites and made minor changes to quotes so that they were not searchable. - Some researchers advocate the use of aggregated quotations (Bond et al., 2013) or composite accounts (Markham, 2012) that represent the meaning expressed in multiple quotations, without directly quoting. These prevent traceability and protect privacy and anonymity/pseudonymity of individuals and online communities. Netnography – Kozinets, 2002 - Ethnography online! - Advantages : access to a vast number of online communities with participants spread all across the world. Can be an inexpensive method for conducting international research. Longitudinal studies possible. Important to understand what is happening in these spaces – research can lag behind the world when it comes to technological developments. - Disadvantages : Kozinets explains that you can only really claim to be doing netnography if you reveal your presence and research purpose to all participants. This can be difficult/impossible in some online situations. Consider doing a netnography on Twitter – how do you add a research disclosure statement in every tweet/interaction? What does informed consent look like here? Important General Reminders - Always ask permission before recording and explain what the purpose of recording is (e.g., to make a transcript, which will be de-identified) - De-identify your transcripts (no names, disguise details that make it obvious who it is) - Paraphrase quotes that may easily link back to the person who stated it - Only out interviews are transcribed, de-identity the transcript - Think about where and for how long you are storing your data (e.g., are your interviews still on your phone? What if it is lost/stolen?) - Must cite academic literature! Use quote marks “” and page numbers for direct quotes - Be careful when inviting friends to take part in your study – don’t pressure them and make it clear that it won’t affect your friendship if they decline, otherwise it’s not really freely given consent - If you learn something interesting about a participant, don’t share this with friends who are not involved in the research project, maintain confidentiality and respect your participants - Keep any promises! E.g., destroy your audio recording, remove identifying features, not name the organization, etc. Writing Up Your Qualitative Research Reporting Qualitative Research Results - Your credibility is at stake : Need to convince the reader that your interpretation is trustworthy : Do this through presentation of results - Not always a straightforward process : A table of numbers shows clearly quantitative results, not so in qualitative : Difficult to condense rich text - Controversies of representation of qualitative research findings : A deductive logic is still prevalent in mainstream journals (Tracy, 2012) : Who are you writing for? Multiple audiences Creating a ‘Theorised Storyline A Theorized storyline is the articulation of a plot that relates the field and academic worlds via “literature-based” ideas that cohere with our field engagement. An example of a theorized storyline might read as follows: - “Y” is an important phenomenon. The extant literature thinks “x” about “y.” - However, “y” is in reality more complicated than “x” suggests; “x” is not the whole story. - Out study helps us to gain insight “q” which needs to be incorporated in to the understanding of “y.” - If we adopt these theoretically relevant insights, we can better explain “y” or see it differently from before. - This richer view of “y” is important because what we matter Common Mistakes in Reporting Qualitative Research Pratt, 2009 1. Telling about data, not showing it 2. Showing too much data, and not interpreting it 3. Using deductive “shorthand” 4. Quantifying qualitative data 5. Inappropriately mixing inductive and deductive strategies Better Paths for Reporting Qualitative Research – Pratt, 2009 1. Make sure your methods section includes “the basics” 2. Show data – in a smart fashion 3. Think about using organizing figures 4. Think about telling a story 5. Consider “modeling” someone whose style you like who consistently published qualitative work Using Your Data: “Telling, Showing, and Telling” The ‘sandwich approach’ to using your data in the body of your paper: 1) First explain the core idea that will be depicted in the following data 2) Show that data (e.g., a quote or detail from fieldnote) 3) Finally tell more abstractly what the data showed Tips for Writing Up Start in the easy place: explaining ‘what you did’ (research design and methods) - Justify each choice you made and reference academic literature where possible (e.g., sampling) - Cite readings from the course when you are explaining your research design and method choices – include the reasons that the authors give for when and how to use certain methods/approaches (and cite appropriately) - Use tables to give a simple to read overview of what you did Use academic literature, especially in your introduction and in your discussion - You must cite 4 academic articles that help set up your topic in the introduction - Try to identify something that these articles miss or don’t go into enough detail on, which becomes a ‘gap’ that your study has partly addressed - Come back to the literature at the end of your paper to ‘close the circle’ and show what your findings have contributed to academic knowledge Define key terms and concepts using appropriate references - Whenever you use a term that has a specific meaning in the literature you are reading, define it and cite your source for the definition - This should not be a dictionary - If there are several definitions in the literature you have read, either describe this diversity and try to explain why there may not be ‘one definition’, and/or decide on one definition, cite it, and explain why you chose this definition. Think about how you will communicate your findings - Go beyond summarizing each interview! - At least organize your findings into themes - Even better, consider how the different things that you found out relate to one another and the academic literature - Here a data structure or conceptual diagram can be included to communicate your findings visually Also consider the world of ‘practice’ (by this academics mean the ‘real world’ or ‘industry’, where people do things) - Describe the context of your study and explain what the people you studied do, what is important to them (their priorities), and what they struggle with (if you know this) - In the discussion, consider how what you have learned might be of interest or of use to the people you studied – this is referred to as ‘implications for practice’ – give 2 recommendations in your paper that you could imagine passing on to those you studied
