IRB, Content Analysis
advertisement
Working with participants CMPE 131/231 - PSYC 131/223, DANM 231: Human-Computer Interaction ►Research oversight1 is needed Too many clinical trial blunders The future impact of such issues as cloning, gene therapy, genetic engineering, etc. is unknown. ►Less we forget Nuremberg code Tuskegee syphilis study Sri Kurniawan – E2/331 Project-related tips ►History: National Research Act of 1974 Established “National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research” Required establishment of IRBs at institutions receiving the Department of Health and Human Services funding for human subjects research Course contact: srikur@soe.ucsc.edu 1 The Institutional Review Board 1a system for addressing questions of potential risk through guidelines, regulations or other structures The Institutional Review Board ►Roles and responsibility. ►Mandated for all institutions conducting human research. Review research plan to be sure it meets criteria in Federal regulations Confirm there are no unreasonable risks Conduct continuing review Assess suspected or alleged protocol violations. Any study intended to result in publication or public presentation, including classroom projects. Any activity resulting in publication or public presentation, even if it involves only review of existing data that was collected with no intent to publish. Any use of an investigational drug or device. ►Authority Approve, disapprove, or terminate all research. Require modifications to protocols. Require that information the IRB deems necessary is provided to participants. Require documentation of informed consent, or allow waiver of consent. ►Exempt Æ non research Employee evaluation, program evaluation, quality assurance, or other situations where such evaluation is not designed to lead to generalizable knowledge 3 4 What to submit to IRB Types of IRB Review ►Full board review For research involving risk of physical or psychological harm greater than that encountered in daily life, particularly research involving deception, stress, or manipulation ►Expedited review Collection of data through noninvasive procedures, such as weight, blood pressure, flexibility testing, etc) Materials (data, documents, records, or specimens) that are collected solely for non-research purposes. ►Administrative review Research conducted in accepted educational settings Research involving only observation of public behavior Research involving only surveys or interviews 2 5 ► Protocol statement (What is to be done.) ► Consent forms OR ► Assent forms (for children 7-17 years old) ► All personnel involved and their qualifications ► Location for study ► Special populations, if any ► Data collection method – a copy of the questions might be needed ► Recruitment ads ► Source of funding ► Payment to subjects ► Costs to subjects ► Benefits to subjects ► Risks and discomforts ► Confidentiality – how confidentiality will be maintained for records, videotapes, audiotapes, and how records will be destroyed at end of study. 6 Content analysis ►We ran interviews/focus group discussions – what to do with the data? ►A technique used to study written material by breaking it into meaningful units, using carefully applied rules. ►Use objective and systematic coding to produce a quantitative description of the observed material. Can analyze common myths e.g., women are portrayed as inferior to men in the cowboy movies. e.g., graffiti in toilets are heavy on pornographic comments e.g., spam mails promise money ►Human vs. computer coders ►Can often utilize computers Internet searches Automated text search ►Great for extremely large sets of data Personal judgment not part of the process Cheaper and faster than humans ►Humans Useful for coding complex concepts More flexibility Costs more time and money ►Content analysis seeks to avoid confirmation bias the tendency to look for information that confirms our beliefs and ignore information that disconfirms our belief ►Can be used to quantify concepts ►Can also be used in a qualitative way. ►What can be studied Any written material Audio/visual information ►Useful for 3 types of research Problems involving a large volume of test Research from afar or in the past Revealing themes difficult to see with casual observation. Steps in content analysis 1. 2. 3. 4. Define problem Select the media that will be used Derive coding categories Sampling strategy – (every 10th page, every other sentence?) 5. Choose the coders: Humans vs. computer Human is useful for coding complex concepts Computer removes subjectivity Human costs more time and money Multiple coders and interrater reliability are a must 6. Code the material In vivo codes vs. conceptual constructs Established vs. your own codes 7. Analyze the data Latent vs. manifest content Getting started ►What gets counted? # certain words, # pictures, senders/receivers ►What is important for understanding themes? Explicit themes Number of times mentioned Amount of space dedicated ►What is the coding unit of analysis? Word vs. paragraph vs. themes ►What to analyze? Frequency Direction: Positive vs. negative; happy vs. sad Intensity: Strength of message, minor vs. major issues Space: Picture size, amount of time spent, etc. ►Manifest – overt, visible material How many of times a word appears How many times someone is mentioned Highly reliable coding No judgment ►Latent content – symbolic content; semantic analysis Ex. Level of violence Requires judgment Depends on coders prior knowledge, expectations, etc. Often required – writers portray meaning indirectly Lower reliability, increases with training Allows for more flexibility Coding approaches ►Common classes used by virtually anyone in society, e.g. age, gender, mother, father, etc essential in assessing whether certain demographic characteristics are related to patterns that arise from other coding ►Special classes colloquial categories includes jargon of various professions, e.g. petty crime vs. serious crime ►Theoretical classes those that emerge in the course of analyzing the data category labels generally borrowed from special classes their substance is grounded in the data not immediately knowable until observers spend considerable time with the content Content analysis: an example ►Dowler (2004) - Comparing American and Canadian local television crime stories: A content analysis. 1. Identify problem Is American and Canadian television different at portraying crime? 2. Select media Grounded theory ►Theories are empirically grounded into the data. ►Data collection and analysis are combined. ►Cycle – observe data, modify theory, observe data based on theory ►For content analysis, grounded theory can help find the appropriate codes to use. THEORY START induction FACT deduction Check Material (expect something) revision READ AGAIN Manifest Content ►Type of crime Which crime was being reported on Kept 28 categories of crime, not necessarily mutually exclusive ►Local or national story Origin of study Television news programs 3. Derive coding categories (manifest & latent contents) 4. Sampling strategies: equal 100 30-min news in 4 areas (Detroit, Toledo, Toronto and Kitchener) Latent Content ►Reporting of motive Implied or confirmed Ex. Drug-related; gang-related ►Emotive presentation: 3 categories 1.Presentation of fear: ►Words were explicitly stated about fear, e.g., “be advised”; “on the run” 2.Presentation of outrage or sympathy ►Explicit statements made by reporters or interviewees ►E.g., “tragedy”; “devastated”; “savage”; “horrifying” 3.Sensationalism ►Involving famous people ►Comical stories ►Dramatic arrests ►Vivid descriptions – “bizarre” ►Length of story Used stopwatch to measure exact time spent reporting each study ►Stage of crime: Pre-arrest, Arrest, Court, Disposition ►Live footage? ►Firearm reported? ►Was it the lead story? Content Analysis: spam Distribution of Word-counts in <title> Monetization Random words Well-formed sentences stitched together Links to keep crawlers going Distribution of Visible-content ►Spam more likely in pages with more words in title Spam Content Analysis ► size of the page ► static rank ► link depth ► number of dots/dashes/ digits in hostname ► hostname length ► hostname domain ► number of words in the page ► number of words in the title ► fraction of anchor text ► average length of the words ► fraction of visible content Homework ►Evolutionary theory says women will offer (and men will seek) youth, looks, sex appeal while men will offer (and women will seek) age, status, security. ►Go to http://personals.yahoo.com/ and collect 25 from each category of ads. ►Code those ads on the # occurrence of these themes and indicate ads that show themes from opposite genders. ►Code other themes that emerge. ► fraction of top 100, 200, 500, 1000 words in the text ► fraction of text in top 100, 200, 500, 1000 words ► occurrence of strange words ► occurrence of the phrase “Privacy Policy” ► occurrence of the phrase “Privacy Statement” ► occurrence of the phrase “Customer Service” ► occurrence of the word “Disclaimer” ► occurrence of the word “Fax” ► occurrence of the word “Phone” ► occurrence of the word “Copyright”
