Assignment: January 26, 2011 (Class 2)
Readings:
 Heppner et al. Chap. 1 Science and Counseling
 Heppner et al. Chap. 3 Identifying and Operationalizing Research Topics
 Heppner et al. Chap 4 Choosing Research Designs
 One article that you have identified in your area of interest.
A. Chapter 1
1. What are the definitions of ontology, epistemology and methodology as they apply to the philosophy
of science? (Search for definitions of these words above what the text provides).
2. Counseling Psychology operates from a scientist-practitioner model. How do the philosophical
underpinnings of the concepts of - ontology, epistemology and methodology apply to the process of
therapy, that is, the work of the therapist and the client? Identify specific questions related to therapy
that one may address from the perspective of each one of these domains.
3. Describe in a few sentences the most salient characteristics of the four paradigms of science
discussed in chapter 1: Positivism, Post positivism, Constructivism, and Critical Theory.
4. Think in terms of the following “classical” or well established psychological/philosophical theories
applied to psychotherapy: Classical Psychoanalyses, Modern Psychodynamic, Adlerian, Behavioral,
Cognitive Behavioral, Existential, and Feministic approaches. Identify specific aspects of each
approach that seem to be most closely aligned with assumptions of one or more of the four
paradigms of science discussed in chapter 1: Positivism, Post positivism, Constructivism, and
Critical Theory. Indicate in what are your choices based.
5. On pages 15-16 the authors discuss the goals of science in counseling. Briefly describe your research
interests and describe how they relate (or do not relate) to the goals identified by the authors.
Identify goal(s) of your research that may have not been mentioned by the authors.
B. Chapter 3
1.
According to Kerlinger and Lee (2000), what are the components of a testable research question?
2. Answer as many of these questions as you can after you have conducted a literature search and read
some abstracts of relevant articles:
a.
b.
c.
d.
e.
Describe your research interests in general, broad terms.
Identify a research question in your area of interest
Identify the variables/constructs embedded in the question.
Provide conceptual definitions for these variables.
Provide operational definitions for these variables (refers to methods of assessment of
construct(s); e.g. paper-and-pencil instrument; coded observations).
C. Chapter 4
Select one of the empirical studies that you identified in your literature search and describe for each research
question which of the four research designs described in this chapter it utilized (Descriptive: Laboratory or
Field; Experimental: Laboratory or Field). Describe the characteristics of the study that led you to select that
design over the other three.
Page 1
Assignment February 2, 2011 (Class 3
Readings:
 Heppner et al. Chapter 10 Quantitative Descriptive Designs pp. 224-233 and 243249 (skip the section on Classification or Data Reduction Research Designs)
 Linear Regression Handout (skip the text about moderation analyses)
 Asner-Self & Marotta (2005) study
1. What are the main differences between studies that use these three types of descriptive designs:
survey versus expo facto design versus corelational? What kind of descriptive design(s) best
describes each of the research questions examined in Asner-Self & Marotta (2005)? Why?
2. After reading the Introduction to the Asner-Self & Marotta (2005) article answer the following
questions. According to the authors:
 What is known regarding their topic of study?
 What is not known?
 How does their study attempt to fill the gap?
3. List the preliminary and main research questions examined in the study. For each research question,
identify the names of the predictor variable(s) (independent variable) and the criterion (dependent)
variables.
4. Why do Heppner et al. (07) prefer to label the variables in correlation research predictor/criterion
rather than independent/dependent?
5. With the help of the Linear Regression Handout, interpret the findings reported in Table 2 of the
Asner-Self & Marotta (2005) study:
a. What type of regression analyses did they use in each of the three regressions? Simultaneous,
Stepwise, or Hierarchical? How do you know?
b. Indicate for each of the three criterion (dependent) variables, how much variance did the set of
three predictor variables shared with each of the three criterion variables. Which parameter from
Table 2 gives you that information?
c. List the predictor variables that contributed unique variance to each of the three criterion
variables. Which parameter from Table 2 gives you that information?
6. By looking at the column labeled unstandardized coefficients b on Table 2 of Asner-Self &
Marotta’s (2005) article, can we say that identity confusion contributes more variance to depression
than mistrust? Yes or no and Why?
Note: the p column on Table 2 of Asner-Self & Marotta’s (2005) article provides the p values for
the numbers listed under the column labeled unstandardized coefficients b.
Page 2
Assignment February 9, 2011 (Class 4)
Readings:
 Heppner et al. Chapter 10 Quantitative Descriptive Designs -- pp. 249-255
(moderation/mediation analyses)
 Linear Regression Handout
 Pyrczak & Bruce Chapters 1, 2, and 3
 Corning (2002)
1.
From the Introduction to the Corning (2002) article answer the following questions for Study 1:
 What is known regarding their topic of study?
 What is not known?
 How does their study attempt to fill the gap?
2. Indicate for Corning’s study 1 and study 2 the names of the predictor variable(s) (independent variable),
the moderator variables and the criterion (dependent) variables.
3. Examine the research questions/hypotheses included in the Corning article study 1and critique them in
light of Pyrczak and Bruce Chapters 1,2 ,3; identify specific guidelines they follow and do not follow
4. In regression analyses it is best to have predictor variables that (a) are moderately correlated with the
criterion variable, and (b) are not highly correlated among themselves (to avoid multi-collinearity
problems). Examine Table 1 in Corning (2002). What is the range of correlations of (a) predictor
variables to each other and (b) of the predictor variables to the criterion variables? To what extent do you
think the two conditions described in the first sentence above are met by the data of study 1 and study 2?
5. Is it appropriate to conclude from the results displayed in Table 1 that among college women personal
self-esteem is more strongly related than collective self-esteem to depression and somatization? Explain.
6. Examine the first row of Table 2 in the Corning article: In step 1, how much variance in Depression was
explained by the combined variables Perceived Discrimination and Personal Self-esteem? How much
additional variance in Depression was explained by the interaction effect (Discrimination X Personal
Self-esteem) entered in the 2nd step? Which parameter gives you that information?
7. Examine the 2nd and 3rd rows of Table 2:
(a) Did personal self-esteem moderate the relation of perceived discrimination to anxiety? How do
you know the answer from the information provided in Table 2?
(b) Did personal self-esteem moderate the relation of perceived discrimination to somatization? Can
you answer this last question by examining Table 2?
8. In study 2, the interaction effect was statistically significant in the three regression analyses. Examine
panel B on page 122. By looking at the lines in the graph, state what you can conclude about the relation
between perceived discrimination and depression for people who differ in level of collective self-esteem.
9. Complete Exercise 1 for Pyrczak & Bruce Chapter 1 and also indicate
 if each variable is categorical or continuous and
 which analyses would you use to examine each hypothesis.
Page 3
Assignment February 16, 2011 (Class 5)
Readings:
 An empirical article in your area that you consider is well written
 Heppner et al. Chap 7 pp. 159-162 Section on Factorial Designs
 Heppner et al. Chap. 12 pp.308-311 Section on Status Variables
 Heppner et al. Chap. 13 Dependent Variable (312-322) -up to Factor Analyses
 Heppner et al. Chap. 22 pp.546-48
 Pyrczak and Bruce Chapter 8 Methods
A. Corkin (2008) article
1. What is the difference between “truly independent” variables and “status independent” variables? What
kind of independent variables were used in the in the third research question examined in Corkin et al ?
2. What kind of statistical analyses can be used in studies that use “truly independent” variables; in studies
that use “status independent” variables? What is the main difference between these two types of studies
in terms of the inferences that we can make from their results?
3. Describe in your own words the analyses reported in Table 3. Which results were statistically significant
and which were not. How do you know?
4.
Evaluate the adequacy of the reporting of the psychometric information of the the State-Trait Anxiety
Inventory (STAI) measure in the Methods section of the Corkin et al study according to Heppner et al.
Chap. 22 pp.546-48 and Pyrczak & Bruce guidelines 8.7 – 8.9. In Table 1 included here (a) indicate if
each piece of information is included in the description of the instrument, and (b) evaluate the clarity and
completeness of the information provided (e.g. does the information related to reliability or validity from
previous studies indicate the types of participants scores came from?)
5. Examine the Methods section of the Corkin article in light of Pyrczak & Bruce guidelines 8.1 to 8.6.
B.
Select an empirical article in your area that you consider is well written:
1. Identify the research questions/hypotheses examined in the article in your area and critique them in
light of Pyrczak and Bruce Chapters 1, 2 and or 3; identify specific guidelines they do and do not
follow.
2. Read carefully the introduction/lit review of the article in your area and write out in bullet points
answers to the following three question: (come to class prepared to share your outline).
a. What is known about the area (theory & empirical research);
b. What is not known; and
c. How their study attempts to fill the gap(s).
Page 4
Table 1 (February 16)
Elements to include in describing psychometric properties of instruments in Method sections of research reports
Perceived Discrimination Scale
Collective Self-Esteem Scale
Quality of Information refers to the reporting of the
information (not the adequacy of the coefficients)
1= Adequate; 2= Not Adequate Add Comments
Included
Yes
Quality of information
Comments;
No
1. Description of measure
a. Instrument name
b. Acronym
c. Authors
d. Key References
e. Brief description of construct(s) measured
f. Type of measure (self-report; interview,
behavioral observation)
g. Number and example of items
h. Items response options (e.g. Type of scale,
response options)
i. Factors or subscales and their definitions
j. Scoring options and direction – what a high score
means (total scores, factor scores; higher scores
indicates higher levels of depression)
2. Validity Estimates
a. Convergent and Discriminate Validity
b. Samples on which measure was validated
3. Reliability Estimates
a. Chronbach’s alpha coefficient
- of scores in previous studies
- of scores in current study)
b. Test-retest (if applicable)
Page 5
Included
Yes
No
Quality of information
Comments
Assignment March 2, 2011 (Class 7)
A. Scale Development Readings:
 Heppner et al. Chapter 10 Quantitative Descriptive Designs pp. 233-238 section on
Classification or Data Reduction Research Designs
 Heppner et al. Chap 13 Dependent Variable (322-332) up to Methods of Data Collection
 Heppner et al. Chap. 20 Scale Construction
 Corkin et al. (2008)
B. Class Presentation: Own-Research Conceptualization
 Guide Presentation/Paper Conceptualization & Research Proposal
A. Factor Analysis CDS Spanish Version Corkin et al. 2008
1. What type of extraction did they use to facto analyze the items of the Career Decision Scale?
2. What criteria did they use to decide on the number of factors to extract? How many factor
solutions did they examine?
3. What factor rotation method did they use: orthogonal or oblique. What is the difference?
4. How many items and factors did they retain? What criteria did they use to assign items to
factors?
5. How many of the factors in Table 1 demonstrate simple structure? How do you know?
6. Examine the following three items from the CDS and rate them according to the criteria
proposed by Kline (2005) reported on Heppner et al. Chapter 20:
# 17 I need more information about what different occupations are like before I can make a
career decision
#10. I want to be absolutely certain that my career choice is the “right “ for me, but none of
the careers I know about seem ideal for me
#2 I have decided on a major and feel comfortable with it. I also know how to go about
implementing my choice.
7. The manual of the CDS indicates that the scale can be scored in terms of its dimensions or as a
total career indecision score. In the Corkin study, would have it been appropriate for the authors
to also have examined in the MANOVA (reported on page 88) the effects of career decidedness
groups on overall career indecision as measured by the CDS total score? Explain your answer.
B. Own Research/Conceptualization

Provide a 10 minute presentation with bullet points about the conceptualization of the
questions/hypothesis for your research proposal, following guidelines provided in the:
Guide Presentation/Paper Conceptualization & Research Proposal: 4 students
Page 6
Assignment March 9, 2011 (Class 8)
A.



B.

Between Groups Research Designs Readings:
Heppner et al. Chap. 5 Validity Issues in Research Designs
Heppner et al. Chap 12 Independent variable pp.298-308 (up to status variables)
Elkin et al., (1989)
Class Presentation: Own-Research Conceptualization
Guide Presentation/Paper Conceptualization & Research Proposal
A. Own Research /Conceptualization
 Provide a 10 minute presentation with bullet points about the conceptualization of the
questions/hypothesis for your research proposal, following guidelines provided in the Guide
Presentation/Paper Conceptualization & Research Proposal: 4 students
B. Between Group Research Designs
Based on your readings of Heppner et al. Chapters 5 and 12, address the following issues
regarding the Elkin et al., (1989) study:
1. State the purpose(s) of the study and the research questions they examined.
2. Describe the conditions captured by the independent variable in the study (experimnental
groups).
3. Describe what the researchers did in order to:
a. Limit unwanted differences among treatment conditions, and
b. Enhance the salience of differences among treatment conditions
4. Evaluate the strengths and weaknesses of the study’s design relative to the following threats:
Statistical Conclusion, Internal, Construct and External validity.
 Consider only the specific threats listed In Table 1, which is included in the next
page.

Describe next to each type of threat the information from the study that you used to
evaluate it; indicate with a + or a - if what you identified is a strength or a weakness
 If the issue is not addressed in the Methods and/or Results sections of the article,
indicate so with an X.
Page 7
Table 1 (March 9, 2011)
Validity Issues in Research Design
Elkin et al., (1989) study
A. Threats to Statistical Conclusion Validity
1. Power
2. Fishing and error rates problems
3. Unreliability of treatment implementation
4. Extraneous variance in the experimental setting
5. Heterogeneity of participants (units)
B. Threats to Internal Validity
1. Selection
2. History
3. Attrition
4. Testing Effects
5. Instrumentation
6. Reaction to control group assignment C. Threats to Construct Validity
1. Mono-Operation bias
2. Mono-Method Bias
3. Experimenter Expectancies
4. Novelty and disruption effects
D. Threats to External Validity
Extent of generalizability of treatment to:
1. Persons – Units
2.Treatment Variations
3.Outcome Measures
4. Settings
Page 8
March 30, 2011 (Class 10)
Experimental Studies - Psychotherapy Outcome Readings:
 Heppner et al. Chap. 7 Between Groups …. Designs (pp. 146-166 only)
 Heppner et al. Chap. 18 Outcome Research: Strategies and Methodological Issues
 Elkin et al., (1989)
In the Elkin et al., (1989) study:
1. True or False: In a therapy outcome study designed to examine a psychodynamic intervention
for anxiety in adults, one way to demonstrate that the treatment is efficacious is to show that for
the treatment group the difference between the pre- and post test anxiety scores are statistically
significant (p< .001) at termination and at the six-month follow up. YES or NO. Explain your
answer.
In the Elkin et al., (1989) study:
2. Identify the type of between -group design implemented in the study. List the strengths and
limitations of this design.
3. Which of the seven strategies for conducting outcome research described in Heppner et al.
(2007) pages 427-433 were utilized in the in Elkin et al. (1996) study?
4. What kind of control group did they use? Identify strengths and limitations of the control
group(s) used in the study.
5. How were therapists assigned to (a) the two therapy treatment conditions, and (b) the two
pharmacotherapy conditions? In each case, did the type of therapist assignment enhances or
decreases the probability that differences in outcomes observed may be due to treatment rather
than to therapists’ effects ? Explain your answer.
6. How did they use the pre-test scores in their analyses? Why did they use marital status as a
covariate in the major and secondary analyses?
7. How did they examine the clinical significance of the therapies’ outcomes? Which treatment(s)
showed better outcomes in terms of clinical significance?
8. Describe the findings illustrated on Figure 1. As measured by the HRSD, at termination which
treatment yielded the best outcome for clients with more severe depression? As measured by the
GAScale, at termination which treatment yielded the best outcome for clients with more severe
depression? What seems to be the most effective treatment for those with less severe
depression?
9. What are the implications of the findings of this study for the treatment of depression?
Page 9
Assignment April 20 (Class 13)
A. Class Presentation Own-Research: Proposal students # 7 & 8
 Guide Presentation/Paper Conceptualization & Research Proposal
B. Experimental Studies - Psychotherapy Outcome Readings:
 Jacobson et al., (1996)
 Wampold & Bahti (2004)
Answer these questions in relation to the Jacobson et al (1996) study:
1. Identify the type of between -group design implemented in the study.
2. Which of the seven strategies for conducting outcome research described in Heppner et al. (2007) pages
427-433 were utilized in the in the Jacobson et al. (1996) study?
3. Briefly describe the theoretical arguments for the study and the research questions examined.
4. At the end of the Introduction, Jacobson et al. (1996) state that they are interested in knowing “how
treatment effects change, independently of how well they work.”


How are these two questions different?
What is the implication of each question for the establishment of empirically validated
treatments in psychology?
Answer these questions in relation to the Wampold and Bahti (2006) article:
5.
In which year did the EST movement start in psychology in the US? What was its purpose?
6. What is meant by the term “specificity” in the study of the effectiveness of treatments?
7. Describe the logic or the purposes of the FDA rules for research conducted to approve new drugs. What
are the strengths’ and the limitations of applying such rules for “approving” psychotherapy treatments as
compared to other treatments and/or to medication? Take into account both the logical arguments the
authors make as well as their references to results from research studies of psychotherapy outcome.
8.
According to the authors’ critical analyses of the history of the application of the randomized controlled
clinical trials to education and agriculture, what may explain the lack of importance given to the role of
the psychologist in the type of research advocated by the EBT movement to examine therapy outcome?
9. According to the criteria described in the article’s Appendix,
a. How would you categorize the treatments examined in the Elkins and the Jacobson articles?
b. How would you categorize a family intervention for anorexia that has shown to be superior
to a cognitive behavioral intervention in two randomized clinical studies conducted by the
authors that developed the family intervention?
Page 10
Assignment April 27 (Class 14)
Experimental Studies - Psychotherapy Outcome Readings:
 Jacobson et al., (1996)
 Kazdin (2008)
Answer these questions in relation to the Jacobson et al (1996) study:
1. Did Jacobson et al. used a control group in their study? Was their choice regarding the choice of control
group justified? How so? What are the advantages and disadvantages of using the type of control group
they chose?
2. What type of analyses did they use to compare the efficacy of the various treatments? How did they use
the results of the pre-test in these analyses?
3. Jacobson et al. found that (a) the marital status of patients was not related to therapy outcome and (b) that
there were no therapists’ effects related to outcome. What characteristics of the design of the Jacobson et
al study may help explain these findings?
4. On Table 3 they report results for four subgroups of participants (Total sample, maximum completers,
Completers, & Droputs). For which subgroups the probability of findings statistically significant
treatment effects is higher and for which is it lower? Why?
5. How did Jacobson et al. examine the clinical significance of the study results?
6. State the major findings of the study. What are the implications of the findings of this study (if the
findings are widely replicated) for the treatment of depression with cognitive behavioral therapy?
7. Evaluate the strengths and weaknesses of the study’s design relative to the following threats: Statistical
Conclusion, Internal, Construct and External validity.
a. Consider only the specific threats listed In Table 1, which is included in the next page.
b. Describe next to each type of threat the information from the study that you used to evaluate
it; indicate with a + or a - if what you identified is a strength or a weakness
c. If the issue is not addressed in the Methods and/or Results sections, indicate so with an X.
Regarding Kazdin (2008) article
8. How does he define the terms evidence based treatments (EBT) and evidenced-based practices (EBP) ?
In his views what are the strengths and the problems with each one of these approaches to psychotherapy
work?
9. On page 149 Kazdin states” A central concern about EBT’s involves the generalization of the results
from controlled research to clinical practice.” Identify the concerns Kazdin notes regarding clinical
practice.
10. Identify one of Kazdin’s suggestions to “reduce the gulf between research an practice.” Identify the
strengths and laminations of this suggestions in terms of (a) its practicality (how easy or difficult it may
be to implement, and (b) its chances of result in improved patient care and advances in the knowledge
base.
Page 11
Table 1 (April 27, 2011)
Validity Issues in Research Design
Jacobson et al., (1996) study
A. Threats to Statistical Conclusion Validity
6. Power
7. Fishing and error rates problems
8. Unreliability of treatment implementation
9. Extraneous variance in the experimental setting
10.
Heterogeneity of participants (units)
B. Threats to Internal Validity
7. Selection
8. History
9. Attrition
10.
Testing Effects
11.
Instrumentation
12.
Reaction to control group assignment C. Threats to Construct Validity
5. Mono-Operation bias
6. Mono-Method Bias
7. Experimenter Expectancies
8. Novelty and disruption effects
D. Threats to External Validity
Extent of generalizability of treatment to:
5. Persons – Units
6.Treatment Variations
7.Outcome Measures
8. Settings
Page 12