Assessment Basics:
Implementing and Sustaining a Comprehensive,
Outcomes-Based Assessment Plan
October 19, 2006
Melissa Canady Wargo, Director
Office of Assessment
NCCEA Annual Conference
NCCEA Annual Conference
Waynesville, NC
Questions for today’s session:
•
•
•
•
•
What is assessment and why should
we care about it?
How do I begin?
What do I need to be doing?
How can I use assessment to improve
my program?
Others???
NCCEA Annual Conference
Waynesville, NC
Definition of Assessment
“ … any effort to gather, analyze, or interpret
evidence, which describes institutional,
departmental, or agency effectiveness.” (pp.
3-4) *
Schuh, J. and Upcraft, M. (2001). Assessment Practice in Student Affairs: An
applications manual. San Francisco: Jossey-Bass.
NCCEA Annual Conference
Waynesville, NC
Why Assess Programs & Services?
• To improve programs through informed
decision-making
• To document the quality of service or
extent of student learning for all program
constituents (faculty, administration,
students, alumni, accrediting agencies,
public officials, etc.).
NCCEA Annual Conference
Waynesville, NC
Types of Assessment
• Needs Assessment
• Environmental Assessment
• Cost Assessment
• Outcomes Assessment
NCCEA Annual Conference
Waynesville, NC
Why Outcomes-Based
Assessment?
• Shifts focus from services provided to
results accomplished
• Defines expected end result, assessment
of which allows for focused improvement
• Allows YOU to determine what constitutes
an effective program
NCCEA Annual Conference
Waynesville, NC
What is outcomes-based
assessment?
Interpret
Results
Gather &
Analyze Data
Propose
Improvements
to Program
Program, Division,
University Mission,
Vision, Strategic
Priorities
Develop
Means of
Assessment
Formulate
Outcomes
NCCEA Annual Conference
Waynesville, NC
Implement
Changes
How do I go about assessing my
program?
• Engage all staff in assessment
conversations
• Commit to support assessment activities
• Be willing to use assessment information
to make programmatic changes
NCCEA Annual Conference
Waynesville, NC
How do I begin?
Develop comprehensive assessment plan:
•
•
•
•
•
•
Define program purpose
Set program goals
Articulate intended outcomes
Identify assessment methods
Implement data collection/analysis strategies
Establish process to disseminate and act on
results
NCCEA Annual Conference
Waynesville, NC
Formulating Program Mission
3 Key Questions to Address in Purpose
Statement:
•What is program?
•What does program provide?
•Whom does the program serve?
NCCEA Annual Conference
Waynesville, NC
Sample Mission/Purpose Statement
The mission of the Career Services office is to enhance the
academic, career and personal development of students
by helping them better understand themselves and their
environments, and to gain the knowledge and skills
necessary for academic success, wise career selection,
and satisfying interpersonal relationships. To that end,
Career Services will consult with University faculty and
staff regarding student needs and organizational
development issues; and collect and disseminate
information about changing student needs and the
effectiveness of our services.
NCCEA Annual Conference
Waynesville, NC
Sample Mission/Purpose Statement
University Career Services, an integral part of the
educational process, assists students and alumni in
assessing their career possibilities, setting their personal
goals and achieving their objectives toward becoming
productive citizens in the global community. While assisting
its clients in identifying professional employment
opportunities, University Career Services also provides the
university community with insights into the ever-changing
world of work to help develop realistic ways to better
educate tomorrow’s leaders. (Texas Christian University)
NCCEA Annual Conference
Waynesville, NC
Formulating Program Goals
• Focus on what the program provides to
achieve the mission and outcomes.
• Keep goals brief and general
• Describe an ideal career/coop/internship
program for your type of institution
NCCEA Annual Conference
Waynesville, NC
Sample Program Goals
• To challenge and support students as they
engage in self-exploration.
• To respond to the critical needs of the
University community.
• To provide students with the information,
experience and personal attention they
require to make informed career choices
commensurate with individual strengths
and aspirations.
NCCEA Annual Conference
Waynesville, NC
Formulating Outcomes
Type of Outcomes:
• Program (Operational)
• Learning
NCCEA Annual Conference
Waynesville, NC
Program Outcome
• Statement of what a program intends to
do, achieve, or accomplish through certain
activities or experiences; i.e., what a
program accomplishes for its students,
faculty/staff or institution.
NCCEA Annual Conference
Waynesville, NC
Sample Program Outcomes
•
•
•
•
Placement Rate
# of Coop/Internships
Coop/Internship Participation Rates
# of Employers at Job Fair
NCCEA Annual Conference
Waynesville, NC
Learning Outcome
• Statement of what a constituent should
know, think, be able to do, or value as a
result of participation in/completion of your
program.
NCCEA Annual Conference
Waynesville, NC
Types of Learning Outcomes*
•
•
•
•
•
•
•
•
Complex Cognitive Skills
Knowledge acquisition
Intrapersonal development
Interpersonal development
Practical Competence
Civic Responsibility
Academic Achievement
Persistence
* Adapted from Upcraft, M. (1999). “Assessing
Student Learning.” In Stage, F., Watson, L., &
Terrell, M. (eds.), Enhancing Student Learning:
Setting the campus context.
NCCEA Annual Conference
Waynesville, NC
Learning Outcomes Most Applicable
to Career/Coop/Internship Programs
• Intrapersonal Development: formation of
autonomy, values, identity, self-esteem,
maturity
• Interpersonal development: valuing
diversity, establishing relationships,
relating to others
• Practical Competence: career prep,
time/money management, self-sufficiency
NCCEA Annual Conference
Waynesville, NC
Formulating Outcomes
Think SMART*
• Specific
• Measurable
• Attainable
• Results-Oriented
• Time-Bound
* Adapted from UCF Assessment Handbook
NCCEA Annual Conference
Waynesville, NC
Sample Learning Outcomes
• Career counseling participants will be able to describe
the steps involved in making career decisions.
• Describe at least three job search strategies based on
their career objective.
• Explain to a potential employer how their background
and skills match the employer’s needs.
• Co-op students will exhibit the maturity and
professionalism commensurate with the work
environment.
NCCEA Annual Conference
Waynesville, NC
Group Exercise
Part I
Develop a learning outcome appropriate for
a career/co-op/internship program:
•Think about skills/knowledge/satisfaction
related to student development.
•Remember SMART
NCCEA Annual Conference
Waynesville, NC
Formulating Assessment Methods
• What evidence will allow us to determine
whether the outcome has been met?
• Are those evidence/data attainable?
• How often should they be collected?
• Who is responsible for collection, analysis,
reporting?
• When will the collection, analysis,
reporting occur?
NCCEA Annual Conference
Waynesville, NC
Types of Assessment Measures
• Direct:
– An assessment method designed to directly measure
what a participant knows or is able to do (i.e., requires
demonstration of the skill or knowledge). Most
applicable to learning outcomes.
• Indirect:
– An assessment method that indirectly measures what
a target subject knows, can do or values (i.e., what a
subject perceives or is satisfied with). Most applicable
to process/program outcomes.
NCCEA Annual Conference
Waynesville, NC
Assessment Measures
Evaluation of:
• Experiential activities (e.g. internships, voluntary
activities, etc.)
• Student work samples (e.g., case study responses,
essay responses, etc.)
• Collection of products demonstrating extent of learning
(portfolios)
• Surveys (student, alumni, etc.)
• Exit interviews
• Employment rates
• Retention/graduation data
• Demographics of student population
• Focus groups
NCCEA Annual Conference
Waynesville, NC
Group Exercise
Part 2
Develop two assessment methods
appropriate to the outcome.
Consider:
– What data are needed?
– Where will data be collected?
– Are there mechanisms in place already?
– How often are data collected?
– How will data be disseminated and
discussed?
NCCEA Annual Conference
Waynesville, NC
Discuss your results!
Act on them!
Celebrate your improvements!
NCCEA Annual Conference
Waynesville, NC
How can this process be managed
more effectively?
• Capitalize on activities already in place
• Locate and use resources from around the
University
• Use your time and resources efficiently
NCCEA Annual Conference
Waynesville, NC
Questions/Comments
?
NCCEA Annual Conference
Waynesville, NC