Assessment of Student Affairs
Initiatives for First-Year Students
National Conference on First-Year Assessment
October 12-14, 2008
San Antonio, Texas
Jennifer R. Keup
Director, National Resource Center for
The First-Year Experience & Students in Transition
keupj@mailbox.sc.edu
Goals for Today



Clarify the relationship between FYE,
Student Affairs & Assessment for you
Identify and discuss challenges to FYE
assessment in Student Affairs
Review a Comprehensive Assessment
Model




Discuss Assessment Outcomes
Assessment Audit
Assessment Approaches
Feedback Loops
What is Assessment?


“Assessment is any effort to gather,
analyze, and interpret evidence which
describes institutional, divisional, or
agency effectiveness.”
Effectiveness includes:





Student learning outcomes
Clientele satisfaction
Compliance with professional standards
Comparisons with other institutions
Assessment guides good practice
Introductions and a Question

Introduce yourself:



Name
Institution
Venn Diagram Exercise


Identifying the relationships between
FYE, student affairs, & assessment at
your institution.
Where do you fit in this model on your
campus?
Venn Diagram Exercise:
Identify the Relationships
Assessment
The
First-Year
Experience
Student
Affairs
You
Venn Diagram Example:
Identify the Relationships
Assessment
Me
The FirstYear
Experience
Student
Affairs
What is the assessment process?





Identify expected/desired outcomes.
Plan programs and services that
accomplish outcomes.
Implement programs and
services.
Assess the learning responsive to the
outcomes through sound data
collection & interpretation
Use results to improve learning
opportunities.
Why is this so hard?
Challenges of
First-Year Assessment
in Student Affairs
Barriers to Assessment

Lack of Commitment & Support from
Leadership

Lack of Assessment Expertise

Lack of Resources


Fiscal: $$$

Human: Professional Staff Support
Fear of Results
Let’s “Talk Turkey”
What are some
of the barriers that
are unique to
ASSESSMENT
in Student Affairs?
What are some
of the barriers that
are unique to
ASSESSMENT
of the First-Year
Experience?
What is really going
on for you?
Purposes of FYE Assessment
Gain Information About:







Backgrounds & characteristics of entering
students
Entering students’ needs
Entering students’ satisfaction with
orientation & FYE programs
Outcomes of orientation & FYE programs
Institutional climate for first-year students
Comparisons to national standards
Cost-effectiveness
How do I approach this?
Model of Assessment for
First-Year Initiatives in
Student Affairs
Steps for Assessment
1. Define the problem
2. Determine the purpose of the
3.
4.
5.
6.
study
Determine where to get the
information needed
Determine the best assessment
methods
Determine whom to study
Determine how the data will be
collected
Steps for Assessment (cont.)
7. Determine what instruments will be
used
8. Determine who should collect the
data
9. Determine how the data will be
analyzed
10.Determine the implications of the
study
11.Report the results effectively
Identifying meaningful and
measurable outcomes of Student
Affairs Initiatives for First-Year
Students
1. Define the problem
2. Determine the purpose
of the study
Identifying Outcomes
What are the questions we
need data to answer?
What do we want to know?
How do these answers relate
to FYE initiatives in Student
Affairs?
Types of Outcomes

Program Outcome:



Student Learning Outcomes:



What your program is accomplishing?
What is the degree to which it is being
accomplished?
What the students are learning as a result
of programs & services?
What is the degree of learning attained?
Student Development:

What the students are learning that
advances their abilities to perform
effectively in a life or work situation?
Examples of Outcomes








Persistence
Analytical & critical thinking skills
Knowledge of university requirements
Interpersonal/Communication skills
Creation of a sense of community
Identity exploration and development
Develop educational career
goals/Declare a major
Institutional climate
Examples of Outcomes (cont.)







Cohort/generation characteristics
Multicultural competence
Civic engagement/Develop as
democratic citizens
Ability to identify, seek, & utilize
organizational resources and student
programs
Physical health & emotional wellness
Leadership skills
Moral and Ethical Development
Writing Outcomes

Be clear about what you are assessing




Program Outcome
Student Learning/Development Outcome
Make sure that your outcome is
meaningful
Clearly state what you want students
to know and be able to do


Incremental steps, stages, points of
progress
Indicate the point that represents
fulfillment/success
Outcomes Exercise


Pick an outcome of a program or
service for First-Year Students on
your campus that is managed by a
Student Affairs unit or department.
Define the outcome in specific terms:




What are you assessing?
How is it meaningful?
What do you want students to be able to
know and do?
Key question: Is this outcome
measurable?
Conducting an audit of current
assessment activities of Student
Affairs Initiatives for First-Year
Students
3. Determine where to get
the information needed
What to look for?





Are student data collected?
What is being collected?
By whom?
Where are they housed?
Are these data currently being
utilized?



If so, how are the results communicated?
If not, why not?
What future data collection efforts are
planned?
Potential Sources of Data





Admissions/Registrar’s Data
Institutional participation in national
surveys of first-year students
Program/course evaluations
Utilization statistics of SA
services/programs
Satisfaction surveys for SA
services/programs
Potential Sources of Data (cont.)







Home-grown IR surveys
SA departmental surveys, focus
groups, or student interviews
Accreditation reports
Grants
Strategic planning processes
Self-study processes/CAS
Other sources?
“Who do I know?” Exercise
Write the name of a
“responsible party” from
your campus next to as
many of the “Potential
Sources of Data” items as
possible.
Who do I talk to?

Exercise:



How many were you able to fill in?
Who were the key players?
Resources:





Student Affairs information, assessment
& research specialist
FYE Director (or other equivalent)
IR Director
VC/VP for Student Affairs
Outside agencies
Assessment
methodologies
4. Determine the best
assessment method
6. Determine how the data
will be collected
Methods to Collect Data

Quantitative


Describes and analyzes “what is”
Examples
Surveys
 Analysis of existing data


Qualitative


Collect data about the meaning of events
& activities to the people involved
Examples
Interviews
 Focus Groups
 Observations

Methods to Assess Outcomes

Indirect



Self-reported comments that reveal a
perceived increase in understanding or
appreciation
The perception is usually not verified
through any demonstration of knowledge
acquisition or observation
Direct

Demonstration of abilities, information,
knowledge, etc. as the result of
participation in a program or utilization
of a service
Indirect Assessment Methods

Examples








Satisfaction measures
Program evaluations
Self-ratings of skills
Self-assessment of change
Agreement with statements
Inventories
Informal peer-to-peer conversations
Pro/Con of indirect assessment
methods
Direct Assessment Methods

Examples










Portfolios
Capstones
Performances
Common assignments
Exams
Observations of behavior/performance
Standardized tests
Work samples
Pro/Cons of direct assessment methods
Making indirect methods direct
Closing the loop
10. Determine the implications
of the study
11. Report the results
effectively
Implications

Grounded in Steps 1 & 2: Purpose &
Problem


Have you answered the original
assessment question?
What have we learned about our
outcome of interest?
Effective Reporting: Audience



Who is your client?
Use it as an opportunity to connect
with your list of “responsible parties”
for data.
Who should know about this
information?


If the information can inform practice,
whose practice will it inform?
Know the climate and context of your
university


Venn Diagram
List of Challenges
Effective Reporting:
Structure & Dissemination

Reporting formats:






Executive summary
Visual identity
Leverage information channels that
already exist


Paper
Web
Presentation
Responsible parties for data
Student Affairs Program Review