Portfolio Evaluation: Assessing
Competency
CCPTP conference: 2010 Joint Conference
Michael J. Scheel, Ph.D.
University of Nebraska-Lincoln
Portfolio: The Concept
 “A purposeful collection of student work that tells the story
of the student’s efforts, progress, or achievement in a given
area(s)” (p. 36; Arter & Spandel, 1992).
 Portfolio assessment is used for faculty tenure evaluations;
for some license or diplomate evaluations
 In counseling, beyond performance on a standardized test or
completion of courses, case examples serve to demonstrate
competence (Coleman, 1996).
 “An artist’s portfolio shows not only what the artist knows
but how he or she can put that knowledge into practice over
a range of situations” (p. 219; Coleman, 1996).
Portfolio: The Advantages
 Can demonstrate what one knows that reflects the




complexity of the topic and the integration of skills with
knowledge (Collins, 1992).
Integration of information collected over several contexts
Provides stimulus for discussion in which the student can
present the meaning of an activity or a competency.
Can demonstrate increasing sophistication over time
Stimulates self-reflection because the learner is responsible
for deciding the goal and the content of the assessment
materials.
The portfolio process
1)
2)
3)
4)
5)
Determine the goal
Identify what evidence will demonstrate acquistion of the
goal
Submit a caption (competence statement) about the
importance of the evidence
Prepare a statement (narrative) about how the portfolio as
a whole reflects learning
Fosters a self-reflective process by the learner (Collins,
1992)
Examples of evidence/artifacts
 Client rating of counselor effectiveness; supervisor evaluations
 Self-evaluations; integrative narratives of a content area
 Videotapes of therapy sessions that demonstrate an intervention







approach
Case presentations; Ethical dilemmas presented through a case
A personal ethical decision-making model
Personal theory of change
One page essays on theory orientation, multicultural competence, use of
assessment in practice etc. similar to an APPIC app.
Case presentations and write-ups
Outreach programs presented with powerpoints; presentations at prof.
conferences; publications
Psych. assessments conducted; written reviews of assessment
instruments; reviews of literature
UNL Copsych Portfolio
 Only the counseling psychology program within the Dept. of
Ed. Psych. uses portfolio evaluation with an oral
examination; satisfies the Graduate College comprehensive
exam requirement
 Continuance refining process since 2004; Copsych program
approved latest version in the Fall of 2009 as the only
comprehensive exam method for our program
UNL Portfolio Procedure
1)
2)
3)
meet with your academic advisor to discuss the portfolio
procedure and plans for completing comprehensive examinations
develop a plan during Program of Study committee
meeting in consultation with your doctoral committee regarding the
portfolio procedure; doctoral committee members are invited to
participate; all program core faculty are required to participate
work closely with your academic advisor to assemble portfolio;
submit a completed version to your advisor; portfolio is
comprised of a narrative summary for each competency area; written
narratives are approx. 7 to 10 pp. for each area; the bulk of the
portfolio is comprised of these narrative summaries; a limited number
of accompanying documents/artifacts may be included as
supplementary materials to enhance or further illustrate information
contained in the narrative summaries; E.g., theoretical orientation
essay; mc self-exploration paper, integrated assessment report.
UNL Portfolio Procedure (cont.)
4) Advisor distributes portfolio to each committee member and each
committee member reviews
5) Meeting is held with the committee and student to discuss each
area of the portfolio; Committee members bring questions and
needs for clarification to the meeting after individual reviews of
the portfolio
6) Committee arrives at a consensus decision of ‘pass’ or ‘no pass’
after the conclusion of the meeting;
7) If ‘no pass’ is assigned to a portion of the exam, the student will
be given instructions concerning changes and additions to be
required to bring the area or areas up to a passing level.
Contextual factors
The majority of coursework that fulfills the Counseling
Psychology Core Curriculum should be completed at the
time of Comprehensive Exam (most students complete
portfolio exam in 5th or 6th semester).
2) Doctoral students applying for internship require
notification from the Director of Training that they have
completed their comprehensive exam at the time of their
application. Thus, successful completion of comprehensive
examinations (as well as successful proposal of dissertation)
must be achieved by October 15th of the year in which the
student is applying for pre-doctoral internship.
1)
Portfolio Contents
 Scoring rubrics are provided to assist students in considering
what material to include in their doctoral portfolios.
Specifically, students are expected to demonstrate
competence in the following six areas:
1. Measurement, Assessment, and Psychological Testing
2. Counseling Theories and Practice
3. Career Development
4. Multicultural Issues
5. Ethical, Legal and Professional Issues
6. Area of Emphasis
Scoring Rubric
Counseling Theory and Practice
 Portfolio: Examples of performance indicators
1) Knows constructs and major theories and systems of thought
2) Personal theory of change articulated
3) Counseling interventions
4) Evaluate effectiveness of treatment
Portfolio Area:
Counseling Theory
and Practice
Knows constructs and
major theories and
systems of thought
Benchmarks of
Competency
Foundational
Competencies
• Scientific Knowledge/Methods
• Reflective Practice
• Ind. & Cultural Diversity
• Interdisciplinary Systems
Personal theory of change
Counseling interventions
Evaluate effectiveness of
treatment
Functional Competencies
• Assessment, Diagnosis, and
Case Conceptualization
• Intervention
•Knowledge of Interventions
•Intervention Planning
•Intervention Implementation
Foundational competencies that line up with portfolio
performance indicators
Foundational Comp Benchmarks:
Scientific knowledge/methods: applies evidence in
practice; compares and contrasts theoretical perspectives;
scientific mindedness – articulates issues derived from lit. in
supervision and case conf; reviews scholarly work related to
clinical practice; willingness to present work for scrutiny by
others
Reflective practice self-assessment: awareness of personal
strengths; summarizes lit relevant to client care
Individual and cultural diversity: monitors and applies
knowledge of self as a cultural being in assessment, treatment,
and consultation
Interdisciplinary Systems: demonstrates ability to articulate
the role that others provide in service to clients
Functional Competencies
 Assessment, Diagnosis, and Case Conceptualization:
identifies and conceptualizes symptoms through a
developmental context; formulates treatment plans
 Intervention: Knowledge of interventions
 Intervention: planning
 Intervention: implementation - applies evidence-based
interventions that take into account empirical support,
clinical judgment, and client diversity; clinical skills used in
development of therapeutic relationships; uses clinical
judgment effectively
Portfolio area: Multicultural Counseling
 Examples of performance indicators
1)
2)
3)
4)
5)
Integrated conceptualization of mc competencies
Awareness of personal values  other cultural groups
Knowledge and application of mc theories of psychology
Demonstrates mc skills
Demonstrated plan for continued mc competence
Portfolio Area:
Multicultural
Issues
Integrated
conceptualization of
MC comps
Awareness of
personal values <> other cultural
groups
Benchmarks of
Competency
Foundational
Competencies
•Reflective
Practice/SelfAssessment
•Relationships
•Ind. & Cultural
Diversity
Knowledge and
application of MC
theories of
psychology.
Demonstrates MC
skills
Demonstrate plan
for continued MC
competence
Functional
Competencies
•Assessment, Diagnosis,
and Case
Conceptualization
•Intervention
•Supervision and
Teaching
Foundational Competencies:
Multicultural Counseling
- Reflective practice self assessment: reflective practice self-identifies cultural
identities; articulates attitudes, values, and beliefs toward diverse others
- Relationships: interpersonal relationships: negotiates relationships with ind and
groups that significantly differ from oneself
- Individual Cultural Diversity: Self-awareness (e.g., knowledge of self as a
cultural being; understands cultural identities; critically evaluates feedback and
initiates consultation/supervision over diversity) Applied knowledge (e.g.,
knowledge of cultural diversity science, theory, and contextual issues;
sensitivity to treatment applications with diverse others; understands individual
cultural diversity and APA policies; conceptualizes diversity in appropriate and
effective ways; considers diversity in assessment, diagnosis and case
conceptualization; uses culturally sensitive instruments; seeks consultation;
knowledge of multiple identities in case presentations and impact on these
treatments)
Functional Competencies
 Assessment, diagnosis and case conceptualization:
assessment (e.g., aware and use culturally sensitive
instruments); Integration (e.g., displays know. of assessment
tools specific to a client pop. & specific site)
 Intervention: Planning (e.g., contexually oriented
including diversity); Implementation (e.g., applies evidence
interventions while considering cultural diversity)
 Supervision and teaching: Awareness of factors affecting
quality (e.g., impact of diversity on prof. settings and
supervision; awareness of APA and other policies as
interpreted through cultural and other contexts)
Future directions for portfolio
evaluation development
 Further lining up of portfolio criteria with the competency
benchmarks
 A need for research to validate portfolio assessment as a
method to be used to measure competence (e.g., concurrent
validity studies)
 Need to develop reliable methods of scoring and evaluation
 Any scoring method must recognize and value the holistic
and integrated nature of the performance (Collins, 1992).
Resist the temptation to be reductionistic.
References
 Baltimore, M. L., Hickson, J., George, J. D., Crutchfield, L. B. (1996).




Portfolio assessment: A model for counselor education. Counselor Education and
Supervision, 36(2), 113-121.
Coleman, H. L. K. (1996). Portfolio assessment of multicultural counseling
competency. The Counseling Psychologist, 24(2), 216-229.
Coleman, H. L. K. (1997). Multicultural counseling competencies: Assessment,
Education, and Training, and Supervision. In Pope-Davis, D. B. & Hardin, L. K.
(Eds.) Multicultural aspects of counseling series,Vol. 7. pp. 43-59. Thousand Oaks,
CA: Sage.
Collins, A. (1992). Portfolios for science education: Issues in purpose,
structure, and authenticity. Science Education, 76(4), 451-463.
Van der Schaaf, M. F., & Stokking, K. M. (2008). Developing and validating a
design for teacher portfolio assessment. Assessment and Education in Higher
Education, 33(3), 245-262.
Email address for UNL portfolio
 Theodore.bartholomew@huskers.unl.edu
 mscheel2@unl.edu
For requests of copies of the portfolio