ADVANCED RESEARCH METHODS (Clin Psy 429)
Abbott Hall 12th Floor, Room 1228 Wednesday’s 3:00PM-5:30PM
Professors:
Spring 2014
Office hours: by appointment
Linda A. Teplin, Ph.D. (L-teplin@northwestern.edu) Karen M. Abram, Ph.D. (k-abram@northwestern.edu)
Ph: 312-503-3500; Fax: 312-503-3535
Ph: 312-503-3500; Fax: 312-503-3535
Leah Welty, Ph.D. (Lwelty@northwestern.edu)
Phone: (312) 503-4710; Fax: 312-908-9588
April 2
Introduction
April 9
The Nature of Method
(Hoyle, ch. 1, 2 & 4; Campbell & Fiske, 1959; Fiske & Campbell, 1992)
April 16
Types of Validity: Internal Validity, Statistical Conclusion Validity
(Hoyle, pp. 245-252; Wilkinson & APA Task Force on Statistical Inference, 1999; Gliner et
al., 2001; P. Cohen, 1982; J. Cohen, 1990; Erceg-Hurn, 2008, Simmons et al.,2011;
APS,2012 Neuroskeptic’s Nine Circles of Scientific Hell)
April 23
Present Research Idea in Class
Also, prepare for preproposal (due on May1) and paper by reading (Hoyle, ch. 20;
Zinsser; Bem, 1995; Bloom, 2010; Cullen, 1989; Eichorn and Yankauer,1987; Harlow,1962;
Rosenfield and Hoffman, 2009; Cummings and Rivera, 2011; APA Publications &
Communications Board Working Group, 2008)
April 30
Construct Validity & External Validity
(Hoyle, pp. 252-254, 264-269; Swanson et al., 1990; Binswanger et al., 2010; Makel et al.,
2012; Klein et al., 2012; Ferguson & Heene, 2012; Pashler & Harris, 2012)
PREPROPOSAL DUE
May 7
Measurement
(Hoyle, chapters 5 & 7; Narrow et al. (with Wakefield comment), 2002;
Schinnar et al., 1990; Shaffer et al., 1996)
May 14
Experiments, Quasi-Experimental Studies & Other Research Strategies
(Hoyle, pp. 237-245 and pp. 254-264, and ch. 12, 13, & 14)
Suggested reading: Observational, Archival & Qualitative Research
(Hoyle, ch. 15, 16)
May 21
Avoiding Common Pitfalls in Statistics
Readings to be announced
May 28
Sampling: Types & Sizes
(Hoyle, ch. 8, 10; Swanson, 1993; Khan et al., 2011)
June 4
Human Subjects and Ethics
(Hoyle, ch. 3; Milgram, 1963; Blanck et al., 1992; Fisher, 2000; Turner et al., 2008.
Washburn, 2008). Additional readings to be announced.
June 11
PAPER IS DUE *
*Papers submitted for credit in this course should be uploaded to Blackboard or sent as e-mail attachments to l-teplin@northwestern.edu or
k-abram@northwestern.edu. Submissions of such work will be taken as your agreement that it may be electronically tested for plagiarized content, and
your acceptance of all University regulations concerning academic integrity.
ADVANCED RESEARCH METHODS
Spring 2014
COURSE OBJECTIVE: Learn how to conduct research, specifically, how to develop a research question,
design a project, and draft a full proposal (with literature review, methods, human subjects, strengths and
limitations)
STATEMENT ON GRADING: 80% based on final paper; 20% on class participation
Texts:
RH Hoyle, MJ Harris, CM Judd, Research Methods in Social Relations (7th edition), Wadsworth Publishing,
2002.
W. Zinsser, On Writing Well: The Classic Guide to Writing Nonfiction, NY: Harper Collins, 2006 (seventh
edition).
Articles:
APA Publications and Communications Board Working Group on Journal Article Reporting Standards. (2008).
Reporting standards for research in psychology: Why do we need them? What might they be? American
Psychologist 63:839-851.
Bem, DJ. (1995). Writing a review article for Psychological Bulletin. Psychological Bulletin 118:172-177.
Binswanger, IA, Merrill, JO, Krueger, PM et al. (2010). Gender differences in chronic medical, psychiatric and
substance-dependence disorders among jail inmates. American Journal of Public Health. 100: 476-482.
Blanck, PD, Bellack, AS, Rosnow, RL et al. (1992). Scientific rewards and conflicts of ethical choices in human
subjects research. American Psychologist 47:959-965.
Bloom, HS. (2010). Nine Lessons About Doing Evaluation Research. Remarks on accepting the Peter H. Rossi
Award. Association for Public Policy Analysis and Management Conference, Boston, Nov. 5, 2010.
Campbell, DT and Fiske, DW. (1959). Convergent and discriminant validation by the multitrait-multimethod
matrix. Psychological Bulletin 56:81-105.
Cohen, J. (1990). Things I have learned (so far). American Psychologist 45:1304-1312.
Cohen, P. (1982). To be or not to be: Control and balancing of type I and type II errors. Evaluation and Program
Planning 5:247-253.
Cullen, FT. (1989). Having trouble getting published? Ten lessons for success. The Criminologist 9, 19-21.
Cummings, P and Rivera, FP. (2011). Responding to reviewers’ comments on submitted articles. Archives of
Pediatrics and Adolescent Medicine 156:105-107.
Eichorn, P. and Yankauer, A. (1987). Do authors check their references? A survey of accuracy of references in
three public health journals. American Journal of Public Health 77:1011-1012.
Erceg-Hurn, DM and Mirosevich, VM. (2008). Modern robust statistical methods: an easy way to maximize the
accuracy and power of your research. Amer. Psychologist 63: 591-601.
Ferguson, J and Heene, M. (2012). A Vast Graveyard of Undead Theories: Publication Bias and Psychological
Science's Aversion to the Null. Perspectives on Psychological Science 7(6):555–561.
Fisher, CB. (2000). Preparing successful proposals for institutional review boards: Challenges and prospects for
behavioral scientists. Psychological Science Agenda March/April 2000:12-13.
2
Fiske, DW and Campbell, DT. (1992). Citations do not solve problems. Psychological Bulletin 112:393-395.
Gliner, JA, Morgan, GE and Harmon, RJ. (2001). Measurement reliability. Journal of the American Academy of
Child and Adolescent Psychiatry 40:486-488.
Harlow, HF. (1962). Fundamental principles for preparing psychology journal articles. Journal of Comparative
and Physiological Psychology 55:893-896.
Khan, MR, Epperson, MW, Mateu-Gelabert, P, Bolyard, M, Sandoval, M. and Friedman, SR. (2011).
Incarceration, sex with an STI- or HIV-infected partner, and infection with an STI or HIV in Bushwick,
Brooklyn, NY: A social network perspective. American Journal of Public Health 101:1110-1117.
Klein, O, Doyen, S, Leys, C, Magalhães de Saldanha da Gama, P, Miller, S, Questienne ,L, and Cleeremans, A.
(2012). Low Hopes, High Expectations: Expectancy Effects and the Replicability of Behavioral Experiments.
Perspectives on Psychological Science 7(6):572–584.
Makel, MC, Plucker, JA, and Hegarty, B. (2012). Replications in Psychology Research: How Often Do They
Really Occur? Perspectives on Psychological Science 7(6):537–542.
Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology 67:371-376.
Narrow, WE, Rae, DS, Robins, LN, Regier, DA. (2002). Revised prevalence estimates of mental disorders in the
United States. Archives of General Psychiatry 59:115-123.
Neuroskeptic. (2012). The Nine Circles of Scientific Hell. Perspectives on Psychological Science, 7(6): 643-644.
Pashler, H and Harris, C. (2012). Is the Replicability Crisis Overblown? Three Arguments Examined.
Perspectives on Psychological Science. 7(6):531–536.
Rosenfield, D and Hoffman, SJ. (2009). Snappy answers to stupid questions: An evidence-based framework for
responding to peer-review feedback. Canadian Medical Association Journal 181:E301-E305.
Schinnar, AP, Rothbard, AB, Kanter, R, and Jung, YS. (1990). An empirical literature review of definitions of
severe and persistent mental illness. American Journal of Psychiatry 147:1602-1608.
Shaffer, D, Fisher, P, Dulcan, MK et al. (1996). The NIMH Diagnostic Interview Schedule for Children Version
2.3 (DISC-2.3): Description, acceptability, prevalence rates, and performance in the MECA study. Journal of
the American Academy of Child and Adolescent Psychiatry 35:865-877.
Simmons, JP, Nelson, LD and Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data
collection and analysis allows presenting anything as significant. Psychological Science 22: 1359-1366.
Swanson, JW. (1993). Alcohol abuse, mental disorder and violent behavior: An epidemiologic inquiry. Alcohol
Health & Research World 17:123-132.
Swanson, JW, Holzer, CE, Ganju, VK and Jono, RT. (1990). Violence and psychiatric disorder in the
community: Evidence from the Epidemiologic Catchment Area surveys. Hospital and Community Psychiatry
41:761-770.
Turner, EH, Matthews, AM, Linardatos, E. et al. (2008). Selective publication of antidepressant trails and its
influence on apparent efficacy. New England Journal of Medicine 358:252-260.
Wakefield, JC. (2002). Lowered estimates-but of what? Commentary. Archives of General Psychiatry 59:129130.
Washburn, JJ. (2008). Encouraging research collaboration through ethical and fair authorship: A model policy.
Ethics & Behavior 18:44-58.
Wilkinson, L. and the Task Force on Statistical Inference (1999), Statistical methods in psychology journals:
Guidelines and explanations, American Psychologist 54:595-604.
3
ASSIGNMENTS STUDENTS MUST COMPLETE:
Guidelines for Research Pre-Proposal
Due: April 23rd, 2014
The purpose of the pre-proposal is to allow for constructive feedback on your research question before
engaging in more in-depth planning.
The pre-proposal should contain the following components:
1. Aim(s): Number and list each research aim clearly and, ideally, in question format. For
example, “What are the prevalence rates of psychiatric disorders among youth in detention?”
2. Justification: In outline form, succinctly explain the key omissions in the literature which your
project intends to address.
3. Study Design: Briefly list your ideas for each component (see below). These decisions need not
be final, but should demonstrate how you are thinking through your research plan.
a.
b.
c.
d.
e.
f.
brief overview of your design (generally one paragraph)
data collection site(s)
type of sample, including inclusion and exclusion criteria
sampling procedures
data collection procedures
measures
4. Key Variables: For each research aim list:
a. Key dependent variable(s) and the variable structure. Are your dependent variables yes/no
variables or otherwise categorical? ordinal? interval/ratio?
b. Key independent variable(s) and the variable structure. Are your independent variables
yes/no variables or otherwise categorical? ordinal? interval/ratio?
c. Control variables, mediators, moderators (or your thinking so far), and the variable
structure. Are these variables yes/no variables or otherwise categorical? ordinal?
interval/ratio?
4
RESEARCH PROPOSAL
REQUIRED COMPONENTS
DUE: JUNE 11, 2014
1. Provide the context for your study. Start with the most general statements and end with the most specific.
2. State Your Research Questions. What do you intend to accomplish and/or what hypotheses will be
tested?
3. Justify need for your study by critiquing the literature. Provide background for your proposal. Critically
evaluate existing knowledge in the area. Delineate the key omissions in the literature which your project
will address. Provide a table summarizing the literature. Use your literature table to argue the key
omission in the literature and the need for your study. (One uses a literature table the same one uses
quantitative tables in a results section).
4. Tell the reader why your study is important. You may have a section, “Conceptual Approach,” if
necessary; you may include a figure (chart) outlining the relationship between your variables.
Alternatively, if your topic is related to public health policy, you may include instead a section
summarizing the importance of your research question for guiding public health policy.
5. Specify your methods. Begin by informing the reader of your overall plan. This can be done in less than a
paragraph. You then need to include the following categories of information. In each category, you need
to give the reader the rationale for your methodological choice. Be aware (and make the reader aware, if
appropriate) of the potential difficulties and limitations of the proposed procedures.
a) Data-collection site. Describe in detail, including the rationale for your choice. Do not choose a site
only because it is convenient.
b) Participants. Describe the characteristics of the subject population such as age range, sex, ethnic
background, health status. Identify criteria for inclusion and exclusion.
c) Sampling Procedures. Describe your recruitment strategy, including the circumstances under which
consent will be sought and obtained who will seek it, the nature of the information to be provided to
prospective participants, and the method of documenting consent.
d) Sample size. Provide a rationale for your sample size. Although you do not have to do a power
analysis for such techniques as ANOVA, outline the steps you would take to set your sample size, with
appropriate references. For statistical techniques where power analysis is inappropriate (e.g.,
correlational studies), detail the rationale for your sample size.
e) Design. Outline your rationale for choosing your design. If appropriate, diagram your design.
f) Data Collection Procedures. Outline your procedures so that the reader can envision your study.
g) Instrumentation. Describe the data collection protocols you will use and delineate the sequence of
data collection. Identify the source of research material, i.e., whether the data consist of existing
records or will be obtained specifically for your study. If your study incorporates existing measures,
specify the rationale for your choice and the relevant reliability and validity information with
appropriate cites. If the goal of your study is such that an adequate instrument does not yet exist, you
do NOT need to develop a measuring instrument. However, include in your proposal the procedures
you would undertake to develop such an instrument.
5
h) Variables and hypotheses. For each aim, list your key variables, specifying their type (dependent,
independent or “control”). Ideally, list your variables in a table. List your hypotheses along with their
direction, if appropriate, e.g., “the combination of drug treatment and therapy will be more effective in
reducing recidivism than either drug treatment or therapy alone.” You may include your hypotheses
in your table.
i)
Data analysis. Include the means by which the data will be analyzed and interpreted. Specify the type
of data and discuss, in detail, the rationale for your choice of statistical procedures. You do NOT need
to include any statistical calculations. If you wish, you may provide mock tables.
j)
Training. Include information concerning your procedures for training staff responsible for data
collection (or coding if needed) such as interviewers, including acceptable levels of reliability.
k) If needed, summarize the strengths and limitations of your study. You may include these in a 3column table including, for example, the issue (e.g., generalizability) and the strengths (column 2) and
weaknesses (column 3).
6. Human Subjects. Discuss: (a) potential risks to subjects (physical, psychological, social, legal, etc.) and
assess their likelihood and seriousness; (b) ways to protect the subject from risks or minimizing potential
risks. If there are risks, discuss why they are reasonable in relation to the anticipated benefits to the
subjects and/or in relation to the importance of the knowledge resulting from your study. Provide a
consent form.
Please note that your topics may require a different organization. Although the above information must be
included, the subheading structure of your proposal may be somewhat different than this example.
6