advertisement

Experimental Research Experimental Research What is experimental research? Research investigation in which conditions are controlled so that hypotheses can be tested and alternative explanations can be ruled out Research used to make “cause-and-effect” statements (X causes Y) X is the independent (or manipulated or causal) variable Y is the dependent variable Experimental Research Evidence of causality (i.e., X causes Y) Evidence of concomitant variation (X and Y covary) Time order sequence of variables The more of X, the better chance that we will get Y If X causes Y, X must occur before Y Elimination of other possible explanations of why Y occurred Key is to keep all experimental conditions equal Experimental Research Must try to eliminate all possible extraneous influences Possible extraneous influences History - event occurring during the course of an experiment (but not really part of the experimental manipulation) that influences the results Maturation - changes which occur in the experimental unit which occur during the course of an experiment that are due to the passage of time Experimental Research Possible extraneous influences (continued) Testing Effects - changes in experimental unit due to the experiment itself (but not related to the key experimental manipulation Measurement effect -- prior measurements effect the measure of the dependent variable Interaction effects -- subject pays greater attention to stimuli than they normally would because its and experiment Experimental Research Possible extraneous influences (continued) Instrument variation -- changes in the measuring instrument cause changes in Y (i.e, interviewer changes the way in which they ask questions) Selection Bias -- if two groups are compared, this notion suggests that the groups may not have been equivalent to begin with; i.e., a manipulation did not cause Y; group dissimilarities did Experimental Research Possible extraneous influences (continued) Experimental mortality -- respondents in experimental groups were lost during the duration of the experiment Statistical Regression -- Extreme responses move closer to the midpoint during the course of an experiment; subjects after questionning do not want to appear to be extreme Experimental Research Experimental Symbols X = Experimental Treatment (e.g., ad exposure) O = Observation R = Random Assignment of Subjects Pre - Experimental Designs One-Shot Case Study X O Problems -- Selection Bias, Control Group, Mortality, History, etc. Experimental Research Pre-Experimental Designs (Cont’d) One Group Pre-Test/Post-Test Design O1 X O2 Analysis -- O2 - O1 Problems -- History, Maturation, interactive Testing Effect, etc. Static Comparison X O1 O2 Analysis -- O2 - O1 Problems -- Selection Bias, Mortality, etc. Experimental Research True Experimental Designs Before/After with Control Group (R) O1 X O2 (R) O3 O4 Analysis (O4 – O3) – (O2 – O1) Lessens History and Maturation Problems However, interactive testing effect still an issue O2 and O4 may a function of O1 and O3 Experimental Research Experimental Designs (True Experimental Designs) Six Group - Four Study Design (R) O1 X O2 (R) O3 O4 (R) X O5 (R) O6 O2 – O1 compared to O4 – O3 tells if X “worked” (manipulation check) – If yes, go on If O5 > O6 – we have true effects Experimental Research True experiments are very complex Experimental designs that are used are often flawed (or not “True”) Random assignment helps Experimental Research Analyzing Data Typically uses Analysis of Variance (ANOVA) Tests (determines if at least one treatment group mean is different from the others) Ho: Ha: Why not use t-tests? Probability Μ1 = M2 = M3 = … = Mk M1 ≠ M 2 ≠ M3 ≠ … ≠ Mk One comparison at 95% confidence – 95% of rejecting null when it should be rejected Two comparisons -- .95 x .95 = 90.25% Three comparisons -- .95 x .95 x .95 = 86% Like regression Dependent Variable (interval or ratio scaled) Independent Variable (nominal or ordinal scaled) Often group membership Experimental Research (Responses on 1 = Bad; 7 = Outstanding) Treat/Resp Ad 1 Ad 2 Ad 3 1 1 2 3 2 5 2 6 3 5 5 6 4 6 2 6 5 3 2 7 6 6 2 5 Mean 4.33 2.50 5.50 Experimental Research One-Way ANOVA Attempts to partition variance Total Variance (Within Group + Between Group) Within treatment groups (Observation – Group Mean) Between treatment groups (Group Mean – Grand Mean) SSb = Σ nj (Meanj – Meani) 2 MSb = SSb / dfb F-statistic SSw = Σ Σ (x ij – Meani)2 MSw = SSw / df w MSb / MSw With dfb,; df w Follow-up test Tukey (most common in MR) Experimental Research Additional Issues with ANOVA Real World Often used with survey data Two-Way ANOVA One dependent variable Two or more independent variables Simultaneous effects MANOVA Compare means of several groups DV – is interval (or ratio) scaled IV -- categorical Multiple dependent variables ANCOVA Investigate effects after “controlling” for another variable (that is interval or ratio scaled) Experimental Research Compare means of several groups Cool Warm EXAMPLE (Survey Data) Permissive Neglecting Restrictive Authoritarian Indulgent Authoritative Experimental Research DV = Parent Responsibility to Restrict TV Choices for Kids F= 4.42; p = .005 Neglecting (M = 29.5) Indulgent (M = 31.26) Authoritarian (M = 30.26) Authoritative (M = 32.27) Follow-up Tests Authoritative > Authoritarian & Neglecting Indulgent > Neglecting Authoritarian = Neglecting Experimental Research Two-Way ANOVA – Example Main Effects Similar to one-way ANOVA effect (for 2 variables) Interactive Effects Main Effects look null Differences observed only when looking at both factors simultaneously Experimental Research Two-Way ANOVA One dependent variable – Main Effects P N Named Not-Named Experimental Research Two-Way ANOVA One dependent variable – Interaction Effects N P Named Not-Named