Foundations of Research 15. Within-subjects & blocking designs This is a PowerPoint Show Open it as a show by going to “slide show”. Click through it by pressing any key. Focus & think about each point; do not just passively click. © Dr. David J. McKirnan, 2014 The University of Illinois Chicago McKirnanUIC@gmail.com Do not use or reproduce without permission 1 Foundations of Research Within-subjects designs. Previous modules discussed experimental approaches: Participants are assigned to different groups; The unit of analysis is the difference between groups. …and correlational designs: Participants have scores on each of two or more variables, We assess the strength of association between variables, across participants. Here we will review within-subject designs: Participants have scores for each of two or more experimental conditions. 2 Foundations of Research Basic forms of within-subjects designs, 1 Basic forms of within subjects designs; 1. Own control Each participant in control and experimental group. Optimally, order is counter-balanced 2. Reversal designs 3. Repeated measures & Randomized block designs 3 Foundations of Research Basic forms of within-subjects designs, 3 Basic forms of Within subjects designs; 1. Own control 2. Reversal designs Hypothesis: behavior controlled by clearly bounded condition Design: “A – B – A”; impose – withdraw – impose condition 3. Repeated measures & Randomized block designs 4 Foundations of Research Basic forms of within-subjects designs, 2 Basic forms of Within subjects designs; 1. Own control 2. Reversal designs 3. Repeated measures Multiple treatment conditions: each participant gets each treatment. Longitudinal / time sampling: each participant assessed over multiple time periods Randomized block designs; Repeated measure combined with between-groups variable. 5 Foundations of Research Within-subjects & blocking designs Own Control designs Reversal designs Repeated measures & Randomized block designs 6 Foundations of Research Your personal own-control design: The McGurk effect Click the image for a video demonstrating a powerful audio-visual illusion. Click the box to begin. Imagine you are watching these stimuli as if you are a participant in an own-control research design… YouTube: jtsfidRq2tw 7 Foundations of Research This experiment The video presents the syllable “ga” The audio presents “ba” Did the illusion work for The brain “fuses” them so most people perceive “da”. you? This is an audio-visual illusion. The within-subjects design allows us to clearly contrast these stimuli within the same people; Your own experience of the stimuli change across condition. You are in both the control group and the experimental group. 8 Basic experiments: “BetweenSubjects” designs Group 1 Experimental condition /Treatment Observe Foundations of Research Between-subjects designs 9 1 Group 2 2 separate groups: Observed (naturally occurring) or Randomly Assigned to be equivalent Control condition Observe2 Independent Variable: Dependent Variable(s): One group receives the experimental condition or treatment, one does not. Measured in both groups. Clear experimental manipulation; treatment given to only one group. Hypothesis tested by differences between groups. Internal validity: groups must differ only on Independent variable; non-equivalence at baseline = confound Statistical power requires large number of subjects. Why do “Within – Subjects” designs? Foundations of Research Within Ss designs Many research questions address contrasts between different states within one person Alcohol v. non-alcohol use aggression, risk Learning condition v. recall state “State dependent learning” Many studies address change over time Behavioral or biomedical intervention studies “Natural history” or cohort studies Practical efficiency of within Ss designs Powerful contrasts within a participant; less within-group (error) variance. Studies that require rare or costly participants. 10 Foundations of Research 11 Pre- Post- designs Non-experimental within-subjects approach Single Group Baseline (Observe1) All participants are assessed at baseline Single Group Assess smoking rate Experimental Manipulation Follow-up (Observe2) All participants then get the Experimental intervention and follow-up measurement. 10-week smoking cessation program Follow-up smoking rate Foundations of Research Single Group 12 Pre- Post- designs Baseline (Observe1) All participants are assessed at baseline Experimental Manipulation Follow-up (Observe2) All participants then get the Experimental intervention and follow-up measurement. Hypothesis is tested by change from baseline This is a main effect analysis “Groups” = baseline v. follow-up scores Possible confounds? History Maturation Regression… Foundations of Research 13 “Own Control” True experiment with within-subjects design Single Group Control Condition Observe1 All participants get the Control condition and measurement Single Group Learning under quiet conditions Recall task Experimental Condition Observe2 All participants then get the Experimental intervention and measurement. Learning under ‘white noise’ condition Recall task Foundations of Research Single Group 14 “Own Control” Control Condition Observe1 Experimental Condition Observe2 Each participant is his own “control group” Hypothesis tested by differences between conditions (Observation1 v. Observation2) within group. Internal validity: eliminate possible confound of group differences at baseline, since there is only one group. Experimental manipulation potentially less clear due to possible carry-over effects. Statistical power increased: requires fewer subjects. Own Control design with counter-balancing Foundations of Research Group 1 Control Condition Observe1 Experimental Condition Observe2 Group 2 Experimental Condition Observe1 Control Condition Observe2 2 groups, naturally occurring or randomly assigned Basic own-control design done twice. Group 1 Learning under quiet conditions Recall task Learning under ‘white noise’ condition Recall task Group 2 Learning under ‘white noise’ condition Recall task Learning under quiet conditions Recall task 15 Foundations of Research 16 Own Control design with counter-balancing Group 1 Control Condition Observe1 Experimental Condition Observe2 Group 2 Experimental Condition Observe1 Control Condition Observe2 Own-control design done twice. Test the Hypothesis: by combining the 2 groups. Internal validity: eliminate confound of group differences at baseline. Lessens (& allows for test of) carry-over effects Statistical power design requires more subjects. Survey example of own-control design Foundations of Research Phenomenon: Treatments for HIV lower the amount of virus in the blood (“viral load”), which may make the person less infectious. Hypotheses: 1. Gay men decide how risky an HIV-infected partner is based on whether the partner has a low viral load. 2. Men who are particularly risk-prone are more likely to decide someone is “safe” via viral load information. Vanable, P.V., Ostrow, D.G., McKirnan, D.J., Tayawaditep, J., & Hope, B.A. (2000). Impact of combination therapies on HIV risk perceptions and sexual risk among HIV-positive and HIV-negative gay and bisexual men. Health Psychology,19(2), 1-12. 17 Foundations of Research Design: Community survey of Chicago gay men 18 IV# 1; Repeated measure: hypothetical partner – The participant reads about each of 2 potential partners: An HIV+ man who is not in treatment An HIV+ men in treatment, with a low viral load – Then rates how risky each partner might be. True independent variable; manipulated by the experimenter. IV #2; Blocking variable: participants’ risk status – Participants describe their history of sexual risk and are categorized by the experimenter as ‘high’ v. ‘low’ risk Measured “blocking” variable only. Foundations of Research 19 Mixed Repeated Measures: data structure Repeated measure: All participants respond to both conditions. Blocking variable: Men “assigned” to high v. low risk groups based on their interview answers. Sexual partner scenario Participant risk background HIV+ man who is in treatment, with a low viral load HIV+ man who is not in treatment Low risk 2, 3, 11, 5, 9, 12, 13, 16… n = 488 2, 3, 11, 5, 9, 12, 13, 16… n = 488 High risk 1, 4, 6, 7, 8, 10, 14, 15… n = 66 1, 4, 6, 7, 8, 10, 14, 15… n = 66 Subject # 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Risk level H L L H L H H H L H L L L H H L Foundations of Research Survey: repeated & blocked analyses, 4 20 Data: A statistical interaction between the repeated measure and the blocking variable. Vanable, P.V., Ostrow, D.G., McKirnan, D.J., Tayawaditep, J., & Hope, B.A. (2000). Impact of combination therapies on HIV risk perceptions and sexual risk among HIV-positive and HIV-negative gay and bisexual men. Health Psychology,19(2), 1-12. Foundations of Research Survey: repeated & blocked analyses, 5 Data: Both risk groups consider an HIV+ partner with low viral load to be safer “Low” viral load information lowers risk perception most for the risky men. Giving all participants both conditions creates a strong contrast (Independent Var.). 21 Foundations of Research Advantages of own-control designs 22 Participant assignment problems eliminated, since there is only one group. No need for random assignment or matching procedures No possible problem with groups differing at baseline. Can use a smaller n than between S’s Half as many participants for simple own-control experiment Very useful for rare or expensive participants More sensitive to experimental effects; less withinsubject (error) variance Foundations of Research Sequencing or order effects: Scores change if participants learn or get practice over different conditions Participants get sensitized to experiment or experimental procedures Cure by counter-balancing order of condition. Disadvantages of own-control designs Takes as many participants as between - group design Participant “burden” Fatigue from multiple experimental tasks Possible drop-out 23 Foundations of Research Within-subjects & blocking designs Own Control designs Reversal designs Repeated measures & Randomized block designs 24 Foundations of Research Basic Reversal design structure “Normal” baseline state Measurement or testing Impose temporary experimental condition Measurement or testing Return to normal state Measurement or testing Examples: Role of incentives in enhancing performance Impact of anti-depressant drug on mood Effect of self-awareness on following social norms 25 Foundations of Research Core Assumptions of reversal designs: Clear stimulus or condition -related hypothesis Hypothesize that behavior is directly tied to a condition or stimulus Clear beginning and ending No “carry-over” effects (due to, e.g., learning, sensitization, etc.) Changes in behavior only last as long as condition is in place Changes can be induced and reversed more than once 26 Foundations of Research Examples of reversal designs Do men in bars change their drinking rate to match that of an attractive woman? A simple interview or questionnaire study my not yield very accurate self-reports. If participants knew they were in an experiment their behavior may change (an experiment may be reactive). This may be tested in a structured, unobtrusive observation study. Provide a model in an actual bar, who systematically modulates her drinking, Record drinking rates of targeted males in her vicinity. 27 Foundations of Research 28 Examples of reversal designs Test effect of, e.g., modeling (observation of attractive experimental confederate) on alcohol consumption. If the model influences participant’s behavior: 50 45 40 35 30 25 20 15 10 5 0 h ig H w Lo h ig H w Lo Model's drinking rate Consumption will increase when the model’s does… Rate goes back down when model’s does. Up again with model, etc.. Foundations of Research 29 Examples of reversal designs Test effect of, e.g., modeling (observation of attractive experimental confederate) on alcohol consumption. This is a basic Reversal Design; Test if a manipulation in the model’s behavior induces change in the participant, 50 45 40 Then test whether that effect can 35 be reversed. 30 25 By inducing then reversing the effect this design can show the outcome to be closely tied to the stimulus. 20 15 10 5 0 h ig H w Lo h ig H w Lo Model's drinking rate Foundations of Research 30 Reversal designs & carry-over Reversal designs can test carry-over effects: 50 50 45 45 40 40 35 35 30 30 25 25 \ 20 15 15 10 10 5 5 0 0 gh Hi w Lo gh Hi w Lo gh Hi w Lo gh Hi w Lo Reversal effect: Modeling controls drinking rate \ 20 Carry-over effect: Drinking rate gradually increases over time no matter what. Foundations of Research Example: single group study with reversal design The Hawthorne Study. 31 The Hawthorne Study is one of the more… Famous, Misunderstood & misused. social research studies ever conducted Context: Hawthorne Electrical Plant (Hawthorne Il.) 1950s era Strong political fears of “Communist conspiracy” Unions seen as ‘fronts’ for international Communism. Working conditions terrible at Hawthorne, as in many companies. OSHA did not exist; worker’s rights largely absent Many Social Psychologists were imbued with the conservative (anti-union) tenor of the times. Foundations of Research Single group reversal design: The Hawthorne Study Context (cont.): Union drive taking place Demands for better working conditions Productivity & quality decreasing Study Purpose 32 Pre-Intervention Baseline assessment 1st Intervention (change lighting) Increase motivation & productivity. Hypothesis Employees simply want attention Any change in the work environment - even a trivial one - leads to change. Intervention Change inadequate lighting. Study structure: Reversal design 1st Follow-up assessment 2nd Intervention (Reversal; change the lighting back) 2nd Follow-up assessment Foundations of Research Single group reversal design: The Hawthorne Study Outcomes Daily assembly line output Findings Output rises after lighting improved (1st follow-up). Output rises again after lighting reversed to initial state. Thus: Any change, even a negative one, “motivates” workers.. Workers respond to simple attention, not real change. Internal validity? Political bias: researchers hired to disprove workers’ claims! Mortality: as part of “union busting” dissatisfied workers fired during study period. Reactive measures: workers fear for job may increase production, not workplace change. Sad legacy of the Hawthorne study The “Hawthorne effect” is commonly cited to discount demands for change, or explain away positive findings of interventions. 33 Foundations of Research Within-subjects & blocking designs Own Control designs Reversal designs Repeated measures & Randomized block designs 34 Foundations of Research 35 Simple repeated measures / time series designs Group Measure1 M2 M3 M4 M5 M6… Group2 Measure1 M2 M3 M4 M5 M6… Group3 Measure1 M2 M3 M4 M5 M6… Examine / describe changes over time in one or more key variables. Describe or test hypotheses about group differences over time. Groups may be assigned, in a true experiment. … intervention groups with long-term follow-up Groups may be measured or naturally occurring. … age, gender or ethnic groups. Longer time-frame yields more valid & interpretable data. Foundations of Research Is attention to childhood obesity causing it to decease? 2003 2012 data Older kids (2 19): no change EXAMPLE Toddlers appear to be doing better. Supports effectiveness of recent infant programs. Click image for the article Longer time frame: 1999 2012 Older kids no still show no change Toddlers only look better because of a spike in 2003. Looking back to 1999 shows a flat line with lots of variance. 36 Is attention to childhood obesity causing it to decease? EXAMPLE Foundations of Research Time series (or longitudinal) designs allow us to sensitively measure change over time. However, results can be sensitive to the specific time frame being tested… 37 Foundations of Research Group Interrupted time series design Measure1 M2 M3 M4 M5 M6 … Intervention or event Test effect of intervention or event on ongoing series of measurements. Intervention may be experimental or observed Policy shift, e.g., educational policy Uncontrolled event; e.g., 9/11/01, Media event Assessments may be experimental or archival Successive cross-sectional surveys Traffic data, clinic or crime reports, test scores 38 Foundations of Research Group Measure1 M2 Multiple baseline 39 Time series designs Demonstrate highly stable effect long-term crime rates disease prevalence economic performance… Show steady rate of change M3 M4 M5 M6 … Intervention or event Hypothesis; tested by: Shift in stable rate after intervention Increase / decrease in rate of change after intervention 40 Example: “Natural” shift in a Baboon culture toward cooperativeness. Question: Foundations of Research Baboons are typically aggressive. Can a baboon troop develop and transmit a learned “culture” of less aggression? EXAMPLE Data: Shutterstock Click for the original article in PLOS Biology. An ongoing, 20+ year observational study of a baboon troop. • Assessments included detailed measures of aggressiveness and other social behaviors, including cooperation. Naturally occurring “Intervention”: Tuberculosis infected Food was mistakenly left in a dumpster. Animals competed with each other to feed on it (a free meal…). The most dominant & aggressive males fed first; They were selectively infected… …and naturally culled from troop. Foundations of Research 41 Example: “Natural” shift in a Baboon culture toward cooperativeness. Question: Baboons are typically It wouldaggressive. be difficult (and unethical) for culltransmit dominant Can a baboonresearchers troop developtoand a males; it may not be of clear and how many to take. learned “culture” lesswhich aggression? EXAMPLE Data: The inadvertent introduction of Click a limited for the original article in PLOS Biology. amount of infected food males to An ongoing, 20+ year observational studyled of adominant baboon troop. “self select” out of the troop. • Assessments included detailed measures of aggressiveness and other social behaviors, including cooperation. Shutterstock Naturally occurring “Intervention”: Tuberculosis infected Food was mistakenly left in a dumpster. Animals competed with each other to feed on it (a free meal…). The most dominant & aggressive males fed first; They were selectively infected… …and naturally culled from troop. 42 Example: “Natural” shift in a Baboon culture toward cooperativeness. Foundations of Research Question: Can a baboon troop develop and transmit a learned “culture” of less aggression? Data: EXAMPLE Ongoing, 20+ year observational study. Shutterstock Click for the original article in PLOS Biology. “Intervention”: Infected food selectively consumed by aggressive males. Analysis: Time Time Series baseline Baseline With naturally occurring “Intervention” variance. Observations extend into the next generation Aggression Cooperation Data simulated Foundations of Research Example: “Natural” shift in a Baboon culture toward cooperativeness. 43 Question: Can a baboon troop develop and transmit a learned “culture” of less aggression? EXAMPLE Data: Shutterstock Ongoing, 20+ year observational study. Click for the original article in PLOS Biology. This data pattern shows the Very long-term data show “Intervention”: behavior of the troop to the troop maintaining the change Infectedafter foodaselectively by aggressive males. naturally consumedchange into the next occurring event. generation: clear evidence Analysis: of culture change. Baseline “Intervention” Observations extend into the next generation Time Aggression Cooperation Data simulated Foundations of Research Example: “Natural” shift in a Baboon culture toward cooperativeness. 44 Question: Studies like this show the major virtue of repeated Can a baboon troop developmeasures: and transmit a learned “culture” of less aggression? This finding is made possible only by virtue of multiple Data: EXAMPLE naturalistic over considerable time. Shutterstock Ongoing, 20+ yearobservations observational study. Click for the original article in PLOS Biology. This data pattern shows the behavior of the troop to change after a naturally occurring event. Baseline Very long-term data show the troop maintaining the change into the next generation: clear evidence of culture change. “Intervention” Observations extend into the next generation Time Aggression Cooperation Data simulated Foundations of Research Randomized block designs Blocking Variable; between - subjects factor “Person” variable; age, gender, ethnicity, etc. Not a “true” IV since people not randomly assigned; Or: Experimental condition; drug dose, treatment, etc. “True” IV with random assignment Repeated measure: within-subjects factor Multiple treatment conditions: each participant is observed in each treatment condition (e.g., high v. low drug dose, different instructions…) Or: Longitudinal / time sampling: measure D.V. over multiple time periods (Cohort studies) 45 46 Within subjects designs; own control, 3 Foundations of Research Repeated measures / randomized block design Group 1 Baseline Measure Control Condition Measure2 M3 M4.. Group 2 Baseline Measure Experimental Condition Measure2 M3 M4.. Assignment Treatment. Primary Randomly or via natural “blocks” Independent Variable. Control group may receive Placebo. Baseline assessment prior to intervention or experimental condition. Follow-up. Long-term assessment of outcome or Dependent Variable. Time may represent 2nd Independent Variable. Example of Repeated Measure Design Foundations of Research Question: Do Gay / bisexual men who use HIV medications as Post-Exposure Prophylaxis [PEP] have more risky sex. Independent variables: Medication use: Between subjects / Measured “blocking” variable: men who request PEP v. those who do not. Study visit [time]: Repeated Measure [within subjects]: Participants are interviewed every 6 months. Dependent variable: % of men reporting risky sex at any given time period. 47 Foundations of Research 48 Unprotected anal intercourse (UAI) with HIV positive and unknown sero-status partners among MSM, by study visit Participants reporting (%) 60 60 Blocking Blocking variable variable 50 50 40 40 PEP users users PEP Non users users Non 30 30 20 20 All men get safer over time 10 10 0 0 Repeated Measure (a Main Effect) 0 0 6 6 12 18 24 30 12 18 24 30 Month of study visit *Adjusted for age, study site, drug use, and education 36 36 OR*(CI) p PEP 1.7(1.2-2.2) .001 Visit 0.97( .96-.99) .001 Foundations of Research 49 Unprotected anal intercourse (UAI) with HIV positive and unknown sero-status partners among MSM, by study visit Participants reporting (%) 60 60 Blocking variable 50 50 40 40 PEP users users PEP Non users users Non 30 30 20 20 Non-users of PEP are generally safer 10 10 (also a Main Effect) 0 0 0 0 6 6 12 18 24 30 12 18 24 30 Month of study visit *Adjusted for age, study site, drug use, and education 36 36 OR*(CI) p PEP 1.7(1.2-2.2) .001 Visit 0.97( .96-.99) .001 Foundations of Research 50 Unprotected anal intercourse (UAI) with HIV positive and unknown sero-status partners among MSM, by study visit Participants reporting (%) 60 60 Blocking variable 50 50 40 40 PEP users users PEP Non users users Non 30 30 20 20 10 10 0 0 Non-users at the end of the study are safest (an Additive Effect; time + group) 0 0 6 6 12 18 24 30 12 18 24 30 Month of study visit *Adjusted for age, study site, drug use, and education 36 36 OR*(CI) p PEP 1.7(1.2-2.2) .001 Visit 0.97( .96-.99) .001 Foundations of Research 51 Unprotected anal intercourse (UAI) with HIV positive and unknown sero-status partners among MSM, by study visit Participants reporting (%) 60 60 Blocking variable 50 50 40 40 PEP users users PEP Non users users Non 30 30 20 20 10 10 0 0 There is no Interaction Effect: The effect of time is the same for both groups. 0 0 6 6 12 18 24 30 12 18 24 30 Month of study visit *Adjusted for age, study site, drug use, and education 36 36 OR*(CI) p PEP 1.7(1.2-2.2) .001 Visit 0.97( .96-.99) .001 HypotheticalHypothetical Example ofblock a design, randomized block randomized 1 design; basic clinical intervention / drug trial Foundations of Research 52 Question: Effectiveness of different doses of an anti-hypertensive drug 2nd questions: Effects of: time, gender, medical status of patients. Population: Hypertensive patients [systolic Bp > 145] Outcome [D.V.] Systolic Blood pressure Blocking variable: Drug dose: placebo v. 2 doses (high / low). This IV carries the main hypothesis. Repeated Measure: Time. Key element of effectiveness: stability x follow-up. 2nd Blocking vars.: Gender, medical status, ethnicity, etc. Foundations of Research 53 Hypothetical randomized block design, 3 Systolic blood pressure Example # 1 (hypothetical data): drug dose & time on systolic Bp, complete sample 180 What does this pattern of data show? Placebo 170 Low Dose 160 High Dose 150 140 130 Main effects? Interaction effects? Hypothesis supported? 120 110 100 Ba s 3 el in e M on 6 th M on 9 th M on 12 th M on th Foundations of Research 54 Hypothetical randomized block design, 4 Systolic blood pressure Example # 2, hypothetical data: drug dose & time on systolic Bp, complete sample 180 What does this pattern of data show? 170 160 150 Main effects? 140 Interactions? Hypothesis supported? 130 Placebo Low Dose High Dose 120 110 100 Ba s 3 el in e M on 6 th M on 9 th M on 12 th M on th Foundations of Research 55 Hypothetical randomized block design, 5 Systolic blood pressure Example # 3, hypothetical data: drug dose & time on systolic Bp, complete sample 180 Main effects? Interactions? Hypothesis? 170 160 150 140 130 120 Placebo Low Dose High Dose 110 100 Ba s 3 el in e M on 6 th M on 9 th M on 12 th M on th Foundations of Research Summary Within subjects designs somewhat common in psychological research; Own control designs: create strong contrast for IV Eliminate problems in creating experimental v. control groups. Very common in bio-medical or public health studies; Most clinical studies are longitudinal; participants followed over time Intervention or experimental treatment is I.V. #1 (blocking or grouping variable). Stability over time is I.V. # 2 (repeated measure) 56